variable size videos. Likewise, CogVideo [22] argues the main reason behind low quality video
generation is the scarcity of good text-video data and tried to leverage pre-trained text to images
models to generate high quality video.
While Phenaki sticks to the same architecture principles, it has major differences with previous work.
Most notably, NUWA, NUWA-Inﬁnity and CogVideo treat videos as a sequence of independent
images. This can lead to poor modeling of dynamics and generate motion artifacts. To combat
this, NUWA-inﬁnity used the previous frame during decoding to combat this. In Phenaki, we go
further and treat videos as a temporal sequence of images which substantially decreases the number
of video tokens given the redundancy in video generation, and results in a much lower training cost.
The auto-regressive nature of the Phenaki also allows us to effectively condition on previous frames
and generates longer videos as detailed in Section 2.
PHENAKI- VARIABLE LENGTH VIDEO GENERATION FROM OPEN DOMAIN TEXTUAL DESCRIPTIONS