Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization
<|reference_start|>Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization: Snow degradations present formidable challenges to the advancement of computer vision tasks by the undesirable corruption in outdoor scenarios. While current deep learning-based desnowing approaches achieve success on synthetic benchmark datasets, they struggle to restore out-of-distribution real-world snowy videos due to the deficiency of paired real-world training data. To address this bottleneck, we devise a new paradigm for video desnowing in a semi-supervised spirit to involve unlabeled real data for the generalizable snow removal. Specifically, we construct a real-world dataset with 85 snowy videos, and then present a Semi-supervised Video Desnowing Network (SemiVDN) equipped by a novel Distribution-driven Contrastive Regularization. The elaborated contrastive regularization mitigates the distribution gap between the synthetic and real data, and consequently maintains the desired snow-invariant background details. Furthermore, based on the atmospheric scattering model, we introduce a Prior-guided Temporal Decoupling Experts module to decompose the physical components that make up a snowy video in a frame-correlated manner. We evaluate our SemiVDN on benchmark datasets and the collected real snowy data. The experimental results demonstrate the superiority of our approach against state-of-the-art image- and video-level desnowing methods.<|reference_end|>
@article{wu2024semi-supervised,
    title={Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts
  and Distribution-Driven Contrastive Regularization},
    author={Hongtao Wu, Yijun Yang, Angelica I Aviles-Rivero, Jingjing Ren,
  Sixiang Chen, Haoyu Chen, Lei Zhu},
    journal={arXiv preprint arXiv:2410.07901},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.07901},
    primaryClass={cs.CV}
}