How to deal with inconsistent touched events

Have you considered using modules (like ZonePlus)?