RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
<|reference_start|>RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents: The ultimate goals of robotic learning is to acquire a comprehensive and generalizable robotic system capable of performing both seen skills within the training distribution and unseen skills in novel environments. Recent progress in utilizing language models as high-level planners has demonstrated that the complexity of tasks can be reduced through decomposing them into primitive-level plans, making it possible to generalize on novel robotic tasks in a composable manner. Despite the promising future, the community is not yet adequately prepared for composable generalization agents, particularly due to the lack of primitive-level real-world robotic datasets. In this paper, we propose a primitive-level robotic dataset, namely RH20T-P, which contains about 33000 video clips covering 44 diverse and complicated robotic tasks. Each clip is manually annotated according to a set of meticulously designed primitive skills, facilitating the future development of composable generalization agents. To validate the effectiveness of RH20T-P, we also construct a potential and scalable agent based on RH20T-P, called RA-P. Equipped with two planners specialized in task decomposition and motion planning, RA-P can adapt to novel physical skills through composable generalization. Our website and videos can be found at https://sites.google.com/view/rh20t-primitive/main. Dataset and code will be made available soon.<|reference_end|>
@article{chen2024rh20t-p:,
    title={RH20T-P: A Primitive-Level Robotic Dataset Towards Composable
  Generalization Agents},
    author={Zeren Chen, Zhelun Shi, Xiaoya Lu, Lehan He, Sucheng Qian, Hao Shu
  Fang, Zhenfei Yin, Wanli Ouyang, Jing Shao, Yu Qiao, Cewu Lu, Lu Sheng},
    journal={arXiv preprint arXiv:2403.19622},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.19622},
    primaryClass={cs.RO cs.CV}
}