An End to End Learning Framework 2 for Video Compression
5 Abstract Traditional video compression approaches build upon the hybrid coding framework with motion compensated prediction 6 and residual transform coding. In this paper, we propose the first end to end deep video compression framework to take advantage of 7 both the classical compression architecture and the powerful non linear representation ability of neural networks. Our framework 8 employs pixel wise motion information, which is learned from an optical flow network and further compressed by an auto encoder 9 network to save bits. The other compression components are also implemented by the well designed networks for high efficiency. All 10 the modules are jointly optimized by using the rate distortion trade off and can collaborate with each other. More importantly, the 11 proposed deep video compression framework is very flexible and can be easily extended by using lightweight or advanced networks for 12 higher speed or better efficiency. We also propose to introduce the adaptive quantization layer to reduce the number of parameters for 13 variable bitrate coding. Comprehensive experimental results demonstrate the effectiveness of the proposed framework on the 14 benchmark datasets.