3.1 TEXT CONDITIONAL VIDEO GENERATION

Currently there is no established benchmark for evaluating text to video methods. This makes com-
paring Phenaki to recent methods such as NUWA [60], CogVideo [22], NUWA-Inﬁnity [59] and
video diffusion models [21] difﬁcult.
Unless speciﬁed otherwise, we train a 1.8B parameter Phenaki model on a corpus of ⇠15M text-
video pairs at 8 FPS mixed with ⇠50M text-images plus ⇠400M pairs of LAION-400M [45] (more

5

Published as a conference paper at ICLR 2023

Table 1. Text to video compar-
isons on Kinetics-400 [26].

Method
T2V [29]
SC [5]
TFGAN [5]
NUWA
Phenaki [0-Shot]

FID
Image #
82.13
33.51
31.76
28.46
37.74

FID
Video #
14.65
7.34
7.19
7.05
3.84

Table 2. Text to video and text to image results highlighting the
importance of image datasets in video models. Text-to-image eval-
uation is done on ⇠40K images of LAION-400M [45].

Text to Video
PHENAKI- VARIABLE LENGTH VIDEO GENERATION FROM OPEN DOMAIN TEXTUAL DESCRIPTIONS