This invention relates generally to visual languages for human computer interfaces, and more specifically to recognizing human visual gestures, as captured by image and video sensors, to develop a visual language for human computer interfaces.
Human Computer Interface (HCI) has been a field of research for more than 30 years with a wide range of applications from innovative ways for scrolling through websites or documents to advanced media manipulation. The recent development of mobile platforms, such as smartphones and tablets, has brought significant innovations in a rapidly developing commercial field, inviting innovative human computer interfaces to enhance user convenience. For example, recently developed mobile communications platforms, such as smartphones and tablets, incorporate multiple image/video cameras, and touch screens with multi-touch sensitivity, and generally avoid traditional keyboards, mince and pencil-like entry devices. Recent gaming devices have added further innovation of incorporating human visual gestures into the gaming system, e.g., multiple sensors including depth sensors, for a computer to understand human body gestures.
One emerging approach to human computer interface is hand gesture recognition, which is the problem of recognizing pre-defined shapes and figures, and any associated motions, formed with a human hand. It is a subfield of gesture recognition, which is the problem of recognizing pre-defined gestures with the human body. A related, but a more challenging example, is recognizing gestures of the human face, or of the human body. Gesture recognition is currently a rapidly developing area of pattern recognition, due to emerging applications in many areas, such as consumer electronics and mobile communications. Gesture recognition is starting to be used in a variety of commercial applications, from video games to controlling consumer electronics products.
There are multiple challenges of existing solutions to human computer interface problems and applications: limited platform size, physical facilities such as keyboards and screen size, limited computing power, and potentially limited bandwidth wireless network connectivity. These challenges stress simplicity and convenience of application design, and put a premium on efficient interfaces. For example, one existing solution recognizes human body motion without any markings, but using multiple sensors including a specialized depth sensor, which makes the human computer interface solution cumbersome and computationally complicated.