Learning Spatial-Semantic Features for Robust Video Object Segmentation
<|reference_start|>Learning Spatial-Semantic Features for Robust Video Object Segmentation: Tracking and segmenting multiple similar objects with complex or separate parts in long-term videos is inherently challenging due to the ambiguity of target parts and identity confusion caused by occlusion, background clutter, and long-term variations. In this paper, we propose a robust video object segmentation framework equipped with spatial-semantic features and discriminative object queries to address the above issues. Specifically, we construct a spatial-semantic network comprising a semantic embedding block and spatial dependencies modeling block to associate the pretrained ViT features with global semantic features and local spatial features, providing a comprehensive target representation. In addition, we develop a masked cross-attention module to generate object queries that focus on the most discriminative parts of target objects during query propagation, alleviating noise accumulation and ensuring effective long-term query propagation. The experimental results show that the proposed method set a new state-of-the-art performance on multiple datasets, including the DAVIS2017 test (89.1%), YoutubeVOS 2019 (88.5%), MOSE (75.1%), LVOS test (73.0%), and LVOS val (75.1%), which demonstrate the effectiveness and generalization capacity of the proposed method. We will make all source code and trained models publicly available.<|reference_end|>
@article{li2024learning,
    title={Learning Spatial-Semantic Features for Robust Video Object Segmentation},
    author={Xin Li, Deshui Miao, Zhenyu He, Yaowei Wang, Huchuan Lu, Ming-Hsuan
  Yang},
    journal={arXiv preprint arXiv:2407.07760},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.07760},
    primaryClass={cs.CV cs.AI}
}