MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints
<|reference_start|>MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints: This work proposes a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. The existing models, which are simplified joint-actuated systems, often produce unnatural motions. To address this, we integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create a new model, MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR is conducted in two separate parts. The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale public datasets and two recent state-of-the-art methods. The results demonstrate that our approach consistently improves the baseline methods both quantitatively and qualitatively.<|reference_end|>
@article{xie2024ms-mano:,
    title={MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints},
    author={Pengfei Xie, Wenqiang Xu, Tutian Tang, Zhenjun Yu, Cewu Lu},
    journal={arXiv preprint arXiv:2404.10227},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.10227},
    primaryClass={cs.CV cs.RO}
}