HARIS: Human-Like Attention for Reference Image Segmentation
<|reference_start|>HARIS: Human-Like Attention for Reference Image Segmentation: Referring image segmentation (RIS) aims to locate the particular region corresponding to the language expression. Existing methods incorporate features from different modalities in a \emph{bottom-up} manner. This design may get some unnecessary image-text pairs, which leads to an inaccurate segmentation mask. In this paper, we propose a referring image segmentation method called HARIS, which introduces the Human-Like Attention mechanism and uses the parameter-efficient fine-tuning (PEFT) framework. To be specific, the Human-Like Attention gets a \emph{feedback} signal from multi-modal features, which makes the network center on the specific objects and discard the irrelevant image-text pairs. Besides, we introduce the PEFT framework to preserve the zero-shot ability of pre-trained encoders. Extensive experiments on three widely used RIS benchmarks and the PhraseCut dataset demonstrate that our method achieves state-of-the-art performance and great zero-shot ability.<|reference_end|>
@article{zhang2024haris:,
    title={HARIS: Human-Like Attention for Reference Image Segmentation},
    author={Mengxi Zhang, Heqing Lian, Yiming Liu, Jie Chen},
    journal={arXiv preprint arXiv:2405.10707},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.10707},
    primaryClass={cs.CV}
}