Cross Pseudo-Labeling for Semi-Supervised Audio-Visual Source Localization
<|reference_start|>Cross Pseudo-Labeling for Semi-Supervised Audio-Visual Source Localization: Audio-Visual Source Localization (AVSL) is the task of identifying specific sounding objects in the scene given audio cues. In our work, we focus on semi-supervised AVSL with pseudo-labeling. To address the issues with vanilla hard pseudo-labels including bias accumulation, noise sensitivity, and instability, we propose a novel method named Cross Pseudo-Labeling (XPL), wherein two models learn from each other with the cross-refine mechanism to avoid bias accumulation. We equip XPL with two effective components. Firstly, the soft pseudo-labels with sharpening and pseudo-label exponential moving average mechanisms enable models to achieve gradual self-improvement and ensure stable training. Secondly, the curriculum data selection module adaptively selects pseudo-labels with high quality during training to mitigate potential bias. Experimental results demonstrate that XPL significantly outperforms existing methods, achieving state-of-the-art performance while effectively mitigating confirmation bias and ensuring training stability.<|reference_end|>
@article{guo2024cross,
    title={Cross Pseudo-Labeling for Semi-Supervised Audio-Visual Source
  Localization},
    author={Yuxin Guo, Shijie Ma, Yuhao Zhao, Hu Su, Wei Zou},
    journal={arXiv preprint arXiv:2403.03095},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.03095},
    primaryClass={cs.CV cs.MM cs.SD eess.AS}
}