Vision-Based Hand Gesture Customization from a Single Demonstration
Hand gesture recognition is becoming a more prevalent mode of human-computer
interaction, especially as cameras proliferate across everyday devices. Despite
continued progress in this field, gesture customization is often underexplored.
Customization is crucial since it enables users to define and demonstrate
gestures that are more natural, memorable, and accessible. However,
customization requires efficient usage of user-provided data. We introduce a
method that enables users to easily design bespoke gestures with a monocular
camera from one demonstration. We employ transformers and meta-learning
techniques to address few-shot learning challenges. Unlike prior work, our
method supports any combination of one-handed, two-handed, static, and dynamic
gestures, including different viewpoints. We evaluated our customization method
through a user study with 20 gestures collected from 21 participants, achieving
up to 97% average recognition accuracy from one demonstration. Our work
provides a viable path for vision-based gesture customization, laying the
foundation for future advancements in this domain.
65cc424ad6c974694f25a16a
Soroush Shahi
65cc424ad6c974694f25a16b
Cori Tymoszek Park
65cc424ad6c974694f25a16c
Richard Kang
65cc424ad6c974694f25a16d
Asaf Liberman
admin_assigned
2024-02-14T12:36:17.172Z
64468634f27198480f19e51a
/avatars/05e8de12dd32bf0798116a42da5a4cda.svg
Asaf Liberman
user
libman
65cc424ad6c974694f25a16e
Oron Levy
admin_assigned
2024-02-14T12:36:22.706Z
6512acd94151fb1fa724123c
/avatars/b994f7184cf921e120b6855382f22903.svg
Oron Levy
user
oronlevy
65cc424ad6c974694f25a16f
Jun Gong
65cc424ad6c974694f25a170
Abdelkareem Bedri
65cc424ad6c974694f25a171
Gierad Laput
Vision-Based Hand Gesture Customization from a Single Demonstration
Hand gesture recognition is becoming a more prevalent mode of human-computer
interaction, especially as cameras proliferate across everyday devices. Despite
continued progress in this field, gesture customization is often underexplored.
Customization is crucial since it enables users to define and demonstrate
gestures that are more natural, memorable, and accessible. However,
customization requires efficient usage of user-provided data. We introduce a
method that enables users to easily design bespoke gestures with a monocular
camera from one demonstration. We employ transformers and meta-learning
techniques to address few-shot learning challenges. Unlike prior work, our
method supports any combination of one-handed, two-handed, static, and dynamic
gestures, including different viewpoints. We evaluated our customization method
through a user study with 20 gestures collected from 21 participants, achieving
up to 97% average recognition accuracy from one demonstration. Our work
provides a viable path for vision-based gesture customization, laying the
foundation for future advancements in this domain.
A method using transformers and meta-learning enables gesture customization with high accuracy from a single demonstration, supporting various gesture types and viewpoints.
transformers meta-learning few-shot learning gesture customization monocular camera hand gestures one-handed two-handed static gestures dynamic gestures viewpoint invariance user study recognition accuracy