Bootstrap Segmentation Foundation Model under Distribution Shift via Object-Centric Learning
<|reference_start|>Bootstrap Segmentation Foundation Model under Distribution Shift via Object-Centric Learning: Foundation models have made incredible strides in achieving zero-shot or few-shot generalization, leveraging prompt engineering to mimic the problem-solving approach of human intelligence. However, when it comes to some foundation models like Segment Anything, there is still a challenge in performing well on out-of-distribution data, including camouflaged and medical images. Inconsistent prompting strategies during fine-tuning and testing further compound the issue, leading to decreased performance. Drawing inspiration from how human cognition processes new environments, we introduce SlotSAM, a method that reconstructs features from the encoder in a self-supervised manner to create object-centric representations. These representations are then integrated into the foundation model, bolstering its object-level perceptual capabilities while reducing the impact of distribution-related variables. The beauty of SlotSAM lies in its simplicity and adaptability to various tasks, making it a versatile solution that significantly enhances the generalization abilities of foundation models. Through limited parameter fine-tuning in a bootstrap manner, our approach paves the way for improved generalization in novel environments. The code is available at github.com/lytang63/SlotSAM.<|reference_end|>
@article{tang2024bootstrap,
    title={Bootstrap Segmentation Foundation Model under Distribution Shift via
  Object-Centric Learning},
    author={Luyao Tang, Yuxuan Yuan, Chaoqi Chen, Kunze Huang, Xinghao Ding, Yue
  Huang},
    journal={arXiv preprint arXiv:2408.16310},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.16310},
    primaryClass={cs.CV}
}