An augmented reality (AR) system hosted and executed on a mobile client enables control of AR objects using gestures. The system receives, from the mobile client, an image from a camera view (e.g., the mobile client's camera) of an environment, where the image depicts a user's hand. The system applies a machine learning model to the received image. The machine learning model identifies a formation of the hand. The system determines to render an AR object based on the identified formation. For example, a user forming a first with his hand may cause an AR ball to move upward within the screen of the mobile client.
GESTURE TRACKING FOR MOBILE RENDERED AUGMENTED REALITY