GraspSAM: When Segment Anything Model Meets Grasp Detection
<|reference_start|>GraspSAM: When Segment Anything Model Meets Grasp Detection: Grasp detection requires flexibility to handle objects of various shapes without relying on prior knowledge of the object, while also offering intuitive, user-guided control. This paper introduces GraspSAM, an innovative extension of the Segment Anything Model (SAM), designed for prompt-driven and category-agnostic grasp detection. Unlike previous methods, which are often limited by small-scale training data, GraspSAM leverages the large-scale training and prompt-based segmentation capabilities of SAM to efficiently support both target-object and category-agnostic grasping. By utilizing adapters, learnable token embeddings, and a lightweight modified decoder, GraspSAM requires minimal fine-tuning to integrate object segmentation and grasp prediction into a unified framework. The model achieves state-of-the-art (SOTA) performance across multiple datasets, including Jacquard, Grasp-Anything, and Grasp-Anything++. Extensive experiments demonstrate the flexibility of GraspSAM in handling different types of prompts (such as points, boxes, and language), highlighting its robustness and effectiveness in real-world robotic applications.<|reference_end|>
@article{noh2024graspsam:,
    title={GraspSAM: When Segment Anything Model Meets Grasp Detection},
    author={Sangjun Noh, Jongwon Kim, Dongwoo Nam, Seunghyeok Back, Raeyoung Kang,
  Kyoobin Lee},
    journal={arXiv preprint arXiv:2409.12521},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12521},
    primaryClass={cs.RO cs.SY eess.SY}
}