Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution
<|reference_start|>Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution: As virtual and augmented reality applications gain popularity, omnidirectional image (ODI) super-resolution has become increasingly important. Unlike 2D plain images that are formed on a plane, ODIs are projected onto spherical surfaces. Applying established image super-resolution methods to ODIs, therefore, requires performing equirectangular projection (ERP) to map the ODIs onto a plane. ODI super-resolution needs to take into account geometric distortion resulting from ERP. However, without considering such geometric distortion of ERP images, previous deep-learning-based methods only utilize a limited range of pixels and may easily miss self-similar textures for reconstruction. In this paper, we introduce a novel Geometric Distortion Guided Transformer for Omnidirectional image Super-Resolution (GDGT-OSR). Specifically, a distortion modulated rectangle-window self-attention mechanism, integrated with deformable self-attention, is proposed to better perceive the distortion and thus involve more self-similar textures. Distortion modulation is achieved through a newly devised distortion guidance generator that produces guidance by exploiting the variability of distortion across latitudes. Furthermore, we propose a dynamic feature aggregation scheme to adaptively fuse the features from different self-attention modules. We present extensive experimental results on public datasets and show that the new GDGT-OSR outperforms methods in existing literature.<|reference_end|>
@article{yang2024geometric,
    title={Geometric Distortion Guided Transformer for Omnidirectional Image
  Super-Resolution},
    author={Cuixin Yang, Rongkang Dong, Jun Xiao, Cong Zhang, Kin-Man Lam, Fei
  Zhou, Guoping Qiu},
    journal={arXiv preprint arXiv:2406.10869},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10869},
    primaryClass={eess.IV cs.CV}
}