Space-Time Video Super-resolution with Neural Operator
<|reference_start|>Space-Time Video Super-resolution with Neural Operator: This paper addresses the task of space-time video super-resolution (ST-VSR). Existing methods generally suffer from inaccurate motion estimation and motion compensation (MEMC) problems for large motions. Inspired by recent progress in physics-informed neural networks, we model the challenges of MEMC in ST-VSR as a mapping between two continuous function spaces. Specifically, our approach transforms independent low-resolution representations in the coarse-grained continuous function space into refined representations with enriched spatiotemporal details in the fine-grained continuous function space. To achieve efficient and accurate MEMC, we design a Galerkin-type attention function to perform frame alignment and temporal interpolation. Due to the linear complexity of the Galerkin-type attention mechanism, our model avoids patch partitioning and offers global receptive fields, enabling precise estimation of large motions. The experimental results show that the proposed method surpasses state-of-the-art techniques in both fixed-size and continuous space-time video super-resolution tasks.<|reference_end|>
@article{zhang2024space-time,
    title={Space-Time Video Super-resolution with Neural Operator},
    author={Yuantong Zhang, Hanyou Zheng, Daiqin Yang, Zhenzhong Chen, Haichuan
  Ma, Wenpeng Ding},
    journal={arXiv preprint arXiv:2404.06036},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.06036},
    primaryClass={cs.CV}
}