  In this paper we focus on landscape animation, which aims to generate
time-lapse videos from a single landscape image. Motion is crucial for
landscape animation as it determines how objects move in videos. Existing
methods are able to generate appealing videos by learning motion from real
time-lapse videos. However, current methods suffer from inaccurate motion
generation, which leads to unrealistic video results. To tackle this problem,
we propose a model named FGLA to generate high-quality and realistic videos by
learning Fine-Grained motion embedding for Landscape Animation. Our model
consists of two parts: (1) a motion encoder which embeds time-lapse motion in a
fine-grained way. (2) a motion generator which generates realistic motion to
animate input images. To train and evaluate on diverse time-lapse videos, we
build the largest high-resolution Time-lapse video dataset with Diverse scenes,
namely Time-lapse-D, which includes 16,874 video clips with over 10 million
frames. Quantitative and qualitative experimental results demonstrate the
superiority of our method. In particular, our method achieves relative
improvements by 19% on LIPIS and 5.6% on FVD compared with state-of-the-art
methods on our dataset. A user study carried out with 700 human subjects shows
that our approach visually outperforms existing methods by a large margin.

Computer Vision and Pattern Recognition