IM Net for High Resolution Video Frame Interpolation
Video frame interpolation is a long studied problem in the video processing field. Recently, deep learning approaches have been applied to this problem, showing impressive results on low resolution benchmarks. However, these methods do not scale up favorably to high resolutions. Specifically, when the motion exceeds a typical number of pixels, their interpolation quality is degraded. Moreover, their run time renders them impractical for real time applications. In this paper we propose IM Net: an interpolated motion neural network. We use an economic structured architecture and end to end training with multi scale tailored losses. In particular, we formulate interpolated motion estimation as classification rather than regression. IM Net outperforms previous methods by more than 1.3dB (PSNR) on a high resolution version of the recently introduced Vimeo triplet dataset. Moreover, the network runs in less than 33msec on a single GPU for HD resolution.
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)