HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution
<|reference_start|>HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution: Transformers have exhibited promising performance in computer vision tasks including image super-resolution (SR). However, popular transformer-based SR methods often employ window self-attention with quadratic computational complexity to window sizes, resulting in fixed small windows with limited receptive fields. In this paper, we present a general strategy to convert transformer-based SR networks to hierarchical transformers (HiT-SR), boosting SR performance with multi-scale features while maintaining an efficient design. Specifically, we first replace the commonly used fixed small windows with expanding hierarchical windows to aggregate features at different scales and establish long-range dependencies. Considering the intensive computation required for large windows, we further design a spatial-channel correlation method with linear complexity to window sizes, efficiently gathering spatial and channel information from hierarchical windows. Extensive experiments verify the effectiveness and efficiency of our HiT-SR, and our improved versions of SwinIR-Light, SwinIR-NG, and SRFormer-Light yield state-of-the-art SR results with fewer parameters, FLOPs, and faster speeds ($\sim7\times$).<|reference_end|>
@article{zhang2024hit-sr:,
    title={HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution},
    author={Xiang Zhang and Yulun Zhang and Fisher Yu},
    journal={arXiv preprint arXiv:2407.05878},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.05878},
    primaryClass={cs.CV}
}