A Simple Image Segmentation Framework via In-Context Examples
<|reference_start|>A Simple Image Segmentation Framework via In-Context Examples: Recently, there have been explorations of generalist segmentation models that can effectively tackle a variety of image segmentation tasks within a unified in-context learning framework. However, these methods still struggle with task ambiguity in in-context segmentation, as not all in-context examples can accurately convey the task information. In order to address this issue, we present SINE, a simple image Segmentation framework utilizing in-context examples. Our approach leverages a Transformer encoder-decoder structure, where the encoder provides high-quality image representations, and the decoder is designed to yield multiple task-specific output masks to effectively eliminate task ambiguity. Specifically, we introduce an In-context Interaction module to complement in-context information and produce correlations between the target image and the in-context example and a Matching Transformer that uses fixed matching and a Hungarian algorithm to eliminate differences between different tasks. In addition, we have further perfected the current evaluation system for in-context image segmentation, aiming to facilitate a holistic appraisal of these models. Experiments on various segmentation tasks show the effectiveness of the proposed method.<|reference_end|>
@article{liu2024a,
    title={A Simple Image Segmentation Framework via In-Context Examples},
    author={Yang Liu, Chenchen Jing, Hengtao Li, Muzhi Zhu, Hao Chen, Xinlong
  Wang, Chunhua Shen},
    journal={arXiv preprint arXiv:2410.04842},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04842},
    primaryClass={cs.CV}
}