Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion
<|reference_start|>Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion: Recently, implicit neural representations (INR) have made significant strides in various vision-related domains, providing a novel solution for Multispectral and Hyperspectral Image Fusion (MHIF) tasks. However, INR is prone to losing high-frequency information and is confined to the lack of global perceptual capabilities. To address these issues, this paper introduces a Fourier-enhanced Implicit Neural Fusion Network (FeINFN) specifically designed for MHIF task, targeting the following phenomena: The Fourier amplitudes of the HR-HSI latent code and LR-HSI are remarkably similar; however, their phases exhibit different patterns. In FeINFN, we innovatively propose a spatial and frequency implicit fusion function (Spa-Fre IFF), helping INR capture high-frequency information and expanding the receptive field. Besides, a new decoder employing a complex Gabor wavelet activation function, called Spatial-Frequency Interactive Decoder (SFID), is invented to enhance the interaction of INR features. Especially, we further theoretically prove that the Gabor wavelet activation possesses a time-frequency tightness property that favors learning the optimal bandwidths in the decoder. Experiments on two benchmark MHIF datasets verify the state-of-the-art (SOTA) performance of the proposed method, both visually and quantitatively. Also, ablation studies demonstrate the mentioned contributions. The code will be available on Anonymous GitHub (https://anonymous.4open.science/r/FeINFN-15C9/) after possible acceptance.<|reference_end|>
@article{liang2024fourier-enhanced,
    title={Fourier-enhanced Implicit Neural Fusion Network for Multispectral and
  Hyperspectral Image Fusion},
    author={Yu-Jie Liang, Zihan Cao, Liang-Jian Deng, Xiao Wu},
    journal={arXiv preprint arXiv:2404.15174},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15174},
    primaryClass={cs.CV}
}