Physical Perception Network and an All-weather Multi-modality Benchmark for Adverse Weather Image Fusion
<|reference_start|>Physical Perception Network and an All-weather Multi-modality Benchmark for Adverse Weather Image Fusion: Multi-modality image fusion (MMIF) integrates the complementary information from different modal images to provide comprehensive and objective interpretation of a scenes. However, existing MMIF methods lack the ability to resist different weather interferences in real-life scenarios, preventing them from being useful in practical applications such as autonomous driving. To bridge this research gap, we proposed an all-weather MMIF model. Regarding deep learning architectures, their network designs are often viewed as a black box, which limits their multitasking capabilities. For deweathering module, we propose a physically-aware clear feature prediction module based on an atmospheric scattering model that can deduce variations in light transmittance from both scene illumination and depth. For fusion module, We utilize a learnable low-rank representation model to decompose images into low-rank and sparse components. This highly interpretable feature separation allows us to better observe and understand images. Furthermore, we have established a benchmark for MMIF research under extreme weather conditions. It encompasses multiple scenes under three types of weather: rain, haze, and snow, with each weather condition further subdivided into various impact levels. Extensive fusion experiments under adverse weather demonstrate that the proposed algorithm has excellent detail recovery and multi-modality feature extraction capabilities.<|reference_end|>
@article{li2024all-weather,
    title={All-weather Multi-Modality Image Fusion: Unified Framework and 100k
  Benchmark},
    author={Xilai Li, Wuyang Liu, Xiaosong Li, Fuqiang Zhou, Huafeng Li, Feiping
  Nie},
    journal={arXiv preprint arXiv:2402.02090},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.02090},
    primaryClass={cs.CV}
}