General-purpose Clothes Manipulation with Semantic Keypoints
<|reference_start|>General-purpose Clothes Manipulation with Semantic Keypoints: Clothes manipulation is a critical skill for household robots. Recent advancements have been made in task-specific clothes manipulation, such as folding, flattening, and hanging. However, due to clothes' complex geometries and deformability, creating a general-purpose robot system that can manipulate a diverse range of clothes in many ways remains challenging. Since clothes are typically designed with specific structures, we propose identifying these specific features like ``left sleeve'' as semantic keypoints. Semantic keypoints can provide semantic cues for task planning and geometric cues for low-level action generation. With this insight, we develop a hierarchical learning framework using the large language model (LLM) for general-purpose CLothes mAnipulation with Semantic keyPoints (CLASP). Extensive simulation experiments show that CLASP outperforms baseline methods on both seen and unseen tasks across various clothes manipulation tasks. Real-world experiments show that CLASP can be directly deployed in the real world and applied to a wide variety of clothes.<|reference_end|>
@article{deng2024general-purpose,
    title={General-purpose Clothes Manipulation with Semantic Keypoints},
    author={Yuhong Deng, David Hsu},
    journal={arXiv preprint arXiv:2408.08160},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.08160},
    primaryClass={cs.RO cs.AI}
}