  Most video platforms provide video streaming services with different
qualities, and the quality of the services is usually adjusted by the
resolution of the videos. So high-resolution videos need to be downsampled for
compression. In order to solve the problem of video coding at different
resolutions, we propose a rate-guided arbitrary rescaling network (RARN) for
video resizing before encoding. To help the RARN be compatible with standard
codecs and generate compression-friendly results, an iteratively optimized
transformer-based virtual codec (TVC) is introduced to simulate the key
components of video encoding and perform bitrate estimation. By iteratively
training the TVC and the RARN, we achieved 5%-29% BD-Rate reduction anchored by
linear interpolation under different encoding configurations and resolutions,
exceeding the previous methods on most test videos. Furthermore, the
lightweight RARN structure can process FHD (1080p) content at real-time speed
(91 FPS) and obtain a considerable rate reduction.

Multimedia;Computer Vision and Pattern Recognition;Image and Video Processing