One way that we get around false triggers is by using some sort of "push-to-gesture" system. This would tell the recognition component that it is time to pay attention. I could imagine using some simpler heuristic that determines if the user is squared off at or at least facing the TV before accepting any gestures.