Adapting Segment Anything Model for Unseen Object Instance Segmentation
<|reference_start|>Adapting Segment Anything Model for Unseen Object Instance Segmentation: Unseen Object Instance Segmentation (UOIS) is crucial for autonomous robots operating in unstructured environments. Previous approaches require full supervision on large-scale tabletop datasets for effective pretraining. In this paper, we propose UOIS-SAM, a data-efficient solution for the UOIS task that leverages SAM's high accuracy and strong generalization capabilities. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder, mitigating issues introduced by the SAM baseline, such as background confusion and over-segmentation, especially in scenarios involving occlusion and texture-rich objects. Extensive experimental results on OCID, OSD, and additional photometrically challenging datasets including PhoCAL and HouseCat6D, demonstrate that, even using only 10% of the training samples compared to previous methods, UOIS-SAM achieves state-of-the-art performance in unseen object segmentation, highlighting its effectiveness and robustness in various tabletop scenes.<|reference_end|>
@article{cao2024adapting,
    title={Adapting Segment Anything Model for Unseen Object Instance Segmentation},
    author={Rui Cao, Chuanxin Song, Biqi Yang, Jiangliu Wang, Pheng-Ann Heng,
  Yun-Hui Liu},
    journal={arXiv preprint arXiv:2409.15481},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15481},
    primaryClass={cs.RO cs.CV}
}