Enhancing Sign Language Teaching: A Mixed Reality Approach for Immersive Learning and Multi-Dimensional Feedback
<|reference_start|>Enhancing Sign Language Teaching: A Mixed Reality Approach for Immersive Learning and Multi-Dimensional Feedback: Traditional sign language teaching methods face challenges such as limited feedback and diverse learning scenarios. Although 2D resources lack real-time feedback, classroom teaching is constrained by a scarcity of teacher. Methods based on VR and AR have relatively primitive interaction feedback mechanisms. This study proposes an innovative teaching model that uses real-time monocular vision and mixed reality technology. First, we introduce an improved hand-posture reconstruction method to achieve sign language semantic retention and real-time feedback. Second, a ternary system evaluation algorithm is proposed for a comprehensive assessment, maintaining good consistency with experts in sign language. Furthermore, we use mixed reality technology to construct a scenario-based 3D sign language classroom and explore the user experience of scenario teaching. Overall, this paper presents a novel teaching method that provides an immersive learning experience, advanced posture reconstruction, and precise feedback, achieving positive feedback on user experience and learning effectiveness.<|reference_end|>
@article{wen2024enhancing,
    title={Enhancing Sign Language Teaching: A Mixed Reality Approach for Immersive
  Learning and Multi-Dimensional Feedback},
    author={Hongli Wen, Yang Xu, Lin Li, Xudong Ru, Xingce Wang, Zhongke Wu},
    journal={arXiv preprint arXiv:2404.10490},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.10490},
    primaryClass={cs.CV}
}