  Video generation is a challenging task that requires modeling plausible
spatial and temporal dynamics in a video. Inspired by how humans perceive a
video by grouping a scene into moving and stationary components, we propose a
method that decomposes the task of video generation into the synthesis of
foreground, background and motion. Foreground and background together describe
the appearance, whereas motion specifies how the foreground moves in a video
over time. We propose V3GAN, a novel three-branch generative adversarial
network where two branches model foreground and background information, while
the third branch models the temporal information without any supervision. The
foreground branch is augmented with our novel feature-level masking layer that
aids in learning an accurate mask for foreground and background separation. To
encourage motion consistency, we further propose a shuffling loss for the video
discriminator. Extensive quantitative and qualitative analysis on synthetic as
well as real-world benchmark datasets demonstrates that V3GAN outperforms the
state-of-the-art methods by a significant margin.

Computer Vision and Pattern Recognition