Learning a Shape-Conditioned Agent for Purely Tactile In-Hand Manipulation of Various Objects
<|reference_start|>Learning a Shape-Conditioned Agent for Purely Tactile In-Hand Manipulation of Various Objects: Reorienting diverse objects with a multi-fingered hand is a challenging task. Current methods in robotic in-hand manipulation are either object-specific or require permanent supervision of the object state from visual sensors. This is far from human capabilities and from what is needed in real-world applications. In this work, we address this gap by training shape-conditioned agents to reorient diverse objects in hand, relying purely on tactile feedback (via torque and position measurements of the fingers' joints). To achieve this, we propose a learning framework that exploits shape information in a reinforcement learning policy and a learned state estimator. We find that representing 3D shapes by vectors from a fixed set of basis points to the shape's surface, transformed by its predicted 3D pose, is especially helpful for learning dexterous in-hand manipulation. In simulation and real-world experiments, we show the reorientation of many objects with high success rates, on par with state-of-the-art results obtained with specialized single-object agents. Moreover, we show generalization to novel objects, achieving success rates of $\sim$90% even for non-convex shapes.<|reference_end|>
@article{pitz2024learning,
    title={Learning a Shape-Conditioned Agent for Purely Tactile In-Hand
  Manipulation of Various Objects},
    author={Johannes Pitz, Lennart R"ostel, Leon Sievers, Darius Burschka and
  Berthold B"auml},
    journal={arXiv preprint arXiv:2407.18834},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.18834},
    primaryClass={cs.RO}
}