RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)
<|reference_start|>RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version): Effective collaboration of dual-arm robots and their tool use capabilities are increasingly important areas in the advancement of robotics. These skills play a significant role in expanding robots' ability to operate in diverse real-world environments. However, progress is impeded by the scarcity of specialized training data. This paper introduces RoboTwin, a novel benchmark dataset combining real-world teleoperated data with synthetic data from digital twins, designed for dual-arm robotic scenarios. Using the COBOT Magic platform, we have collected diverse data on tool usage and human-robot interaction. We present a innovative approach to creating digital twins using AI-generated content, transforming 2D images into detailed 3D models. Furthermore, we utilize large language models to generate expert-level training data and task-specific pose sequences oriented toward functionality. Our key contributions are: 1) the RoboTwin benchmark dataset, 2) an efficient real-to-simulation pipeline, and 3) the use of language models for automatic expert-level data generation. These advancements are designed to address the shortage of robotic training data, potentially accelerating the development of more capable and versatile robotic systems for a wide range of real-world applications. The project page is available at https://robotwin-benchmark.github.io/early-version/<|reference_end|>
@article{mu2024robotwin:,
    title={RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early
  version)},
    author={Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou,
  Lunkai Lin, Zhiqiang Xie, Ping Luo},
    journal={arXiv preprint arXiv:2409.02920},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.02920},
    primaryClass={cs.RO cs.AI cs.CL}
}