  We consider the problem of space-time super-resolution (ST-SR): increasing
spatial resolution of video frames and simultaneously interpolating frames to
increase the frame rate. Modern approaches handle these axes one at a time. In
contrast, our proposed model called STARnet super-resolves jointly in space and
time. This allows us to leverage mutually informative relationships between
time and space: higher resolution can provide more detailed information about
motion, and higher frame-rate can provide better pixel alignment. The
components of our model that generate latent low- and high-resolution
representations during ST-SR can be used to finetune a specialized mechanism
for just spatial or just temporal super-resolution. Experimental results
demonstrate that STARnet improves the performances of space-time, spatial, and
temporal video super-resolution by substantial margins on publicly available
datasets.

Computer Vision and Pattern Recognition