Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations
<|reference_start|>Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations: Bimanual dexterous manipulation is a critical yet underexplored area in robotics. Its high-dimensional action space and inherent task complexity present significant challenges for policy learning, and the limited task diversity in existing benchmarks hinders general-purpose skill development. Existing approaches largely depend on reinforcement learning, often constrained by intricately designed reward functions tailored to a narrow set of tasks. In this work, we present a novel approach for efficiently learning diverse bimanual dexterous skills from abundant human demonstrations. Specifically, we introduce BiDexHD, a framework that unifies task construction from existing bimanual datasets and employs teacher-student policy learning to address all tasks. The teacher learns state-based policies using a general two-stage reward function across tasks with shared behaviors, while the student distills the learned multi-task policies into a vision-based policy. With BiDexHD, scalable learning of numerous bimanual dexterous skills from auto-constructed tasks becomes feasible, offering promising advances toward universal bimanual dexterous manipulation. Our empirical evaluation on the TACO dataset, spanning 141 tasks across six categories, demonstrates a task fulfillment rate of 74.59% on trained tasks and 51.07% on unseen tasks, showcasing the effectiveness and competitive zero-shot generalization capabilities of BiDexHD. For videos and more information, visit our project page https://sites.google.com/view/bidexhd.<|reference_end|>
@article{zhou2024learning,
    title={Learning Diverse Bimanual Dexterous Manipulation Skills from Human
  Demonstrations},
    author={Bohan Zhou, Haoqi Yuan, Yuhui Fu, and Zongqing Lu},
    journal={arXiv preprint arXiv:2410.02477},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.02477},
    primaryClass={cs.RO cs.LG}
}