  We propose $\textbf{VidStyleODE}$, a spatiotemporally continuous disentangled
$\textbf{Vid}$eo representation based upon $\textbf{Style}$GAN and
Neural-$\textbf{ODE}$s. Effective traversal of the latent space learned by
Generative Adversarial Networks (GANs) has been the basis for recent
breakthroughs in image editing. However, the applicability of such advancements
to the video domain has been hindered by the difficulty of representing and
controlling videos in the latent space of GANs. In particular, videos are
composed of content (i.e., appearance) and complex motion components that
require a special mechanism to disentangle and control. To achieve this,
VidStyleODE encodes the video content in a pre-trained StyleGAN $\mathcal{W}_+$
space and benefits from a latent ODE component to summarize the spatiotemporal
dynamics of the input video. Our novel continuous video generation process then
combines the two to generate high-quality and temporally consistent videos with
varying frame rates. We show that our proposed method enables a variety of
applications on real videos: text-guided appearance manipulation, motion
manipulation, image animation, and video interpolation and extrapolation.
Project website: https://cyberiada.github.io/VidStyleODE

Computer Vision and Pattern Recognition