To evaluate the video encoding and reconstruction performance of C-ViViT , we use the Moments-
in-Time (MiT) [33] dataset. MiT contains ⇠802K training, ⇠33K validation and ⇠67K test videos
at 25 FPS. The MiT dataset, in contrast to other publicly available video datasets, is a high quality
balanced dataset with high coverage and density of verbs depicting moments of a few seconds [33].
We compare C-ViViT against per-frame image based encoder-decoders that have been used as video
quantizers for conditional video generation [63, 60, 22, 60, 22, 58]: a ViT [64] and a convolutional
VQ-GAN[14]. The experimental details can be found in the Appendix B.1.
As demonstrated in Table 3, we evaluate the video reconstruction quality using FID [19] and
FVD [50]. Both FID and FVD compare the distribution of generated videos (or images) to the
ground truth distribution. The FID ignores temporal coherency, while the FVD measures how well
PHENAKI- VARIABLE LENGTH VIDEO GENERATION FROM OPEN DOMAIN TEXTUAL DESCRIPTIONS