Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
<|reference_start|>Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning: Teleoperation is a crucial tool for collecting human demonstrations, but controlling robots with bimanual dexterous hands remains a challenge. Existing teleoperation systems struggle to handle the complexity of coordinating two hands for intricate manipulations. We introduce Bunny-VisionPro, a real-time bimanual dexterous teleoperation system that leverages a VR headset. Unlike previous vision-based teleoperation systems, we design novel low-cost devices to provide haptic feedback to the operator, enhancing immersion. Our system prioritizes safety by incorporating collision and singularity avoidance while maintaining real-time performance through innovative designs. Bunny-VisionPro outperforms prior systems on a standard task suite, achieving higher success rates and reduced task completion times. Moreover, the high-quality teleoperation demonstrations improve downstream imitation learning performance, leading to better generalizability. Notably, Bunny-VisionPro enables imitation learning with challenging multi-stage, long-horizon dexterous manipulation tasks, which have rarely been addressed in previous work. Our system's ability to handle bimanual manipulations while prioritizing safety and real-time performance makes it a powerful tool for advancing dexterous manipulation and imitation learning.<|reference_end|>
@article{ding2024bunny-visionpro:,
    title={Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for
  Imitation Learning},
    author={Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan
  Yang, Xiaojuan Qi, Xiaolong Wang},
    journal={arXiv preprint arXiv:2407.03162},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.03162},
    primaryClass={cs.RO cs.CV cs.LG}
}