Open-Vocabulary Part-Based Grasping
<|reference_start|>Open-Vocabulary Part-Based Grasping: Many robotic applications require to grasp objects not arbitrarily but at a very specific object part. This is especially important for manipulation tasks beyond simple pick-and-place scenarios or in robot-human interactions, such as object handovers. We propose AnyPart, a practical system that combines open-vocabulary object detection, open-vocabulary part segmentation and 6DOF grasp pose prediction to infer a grasp pose on a specific part of an object in 800 milliseconds. We contribute two new datasets for the task of open-vocabulary part-based grasping, a hand-segmented dataset containing 1014 object-part segmentations, and a dataset of real-world scenarios gathered during our robot trials for individual objects and table-clearing tasks. We evaluate AnyPart on a mobile manipulator robot using a set of 28 common household objects over 360 grasping trials. AnyPart is capable of producing successful grasps 69.52 %, when ignoring robot-based grasp failures, AnyPart predicts a grasp location on the correct part 88.57 % of the time.<|reference_end|>
@article{van oort2024open-vocabulary,
    title={Open-Vocabulary Part-Based Grasping},
    author={Tjeard van Oort, Dimity Miller, Will N. Browne, Nicolas Marticorena,
  Jesse Haviland, Niko Suenderhauf},
    journal={arXiv preprint arXiv:2406.05951},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05951},
    primaryClass={cs.RO}
}