GPT-Fabric: Folding and Smoothing Fabric by Leveraging Pre-Trained Foundation Models
<|reference_start|>GPT-Fabric: Folding and Smoothing Fabric by Leveraging Pre-Trained Foundation Models: Fabric manipulation has applications in folding blankets, handling patient clothing, and protecting items with covers. It is challenging for robots to perform fabric manipulation since fabrics have infinite-dimensional configuration spaces, complex dynamics, and may be in folded or crumpled configurations with severe self-occlusions. Prior work on robotic fabric manipulation relies either on heavily engineered setups or learning-based approaches that create and train on robot-fabric interaction data. In this paper, we propose GPT-Fabric for the canonical tasks of fabric folding and smoothing, where GPT directly outputs an action informing a robot where to grasp and pull a fabric. We perform extensive experiments in simulation to test GPT-Fabric against prior state of the art methods for folding and smoothing. We obtain comparable or better performance to most methods even without explicitly training on a fabric-specific dataset (i.e., zero-shot manipulation). Furthermore, we apply GPT-Fabric in physical experiments over 12 folding and 10 smoothing rollouts. Our results suggest that GPT-Fabric is a promising approach for high-precision fabric manipulation tasks.<|reference_end|>
@article{raval2024gpt-fabric:,
    title={GPT-Fabric: Smoothing and Folding Fabric by Leveraging Pre-Trained
  Foundation Models},
    author={Vedant Raval, Enyu Zhao, Hejia Zhang, Stefanos Nikolaidis, Daniel
  Seita},
    journal={arXiv preprint arXiv:2406.09640},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.09640},
    primaryClass={cs.RO}
}