Our experimental results demonstrate the VideoPoet’s
state-of-the-art capabilities in generating videos with large
and high-ﬁdelity motions.
In particular, we observe that
through the powerful capabilities of the transformer archi-
tecture, the VideoPoet can be straightforwardly trained on
a multi-task, multimodal generative objective, allowing for
generating consistent and realistic motion driven by text, as
shown in Figure 2, or other prompts.

We found that VideoPoet, a LLM, is capable of zero-shot
video generation. We use the term “zero-shot video gener-
ation” because VideoPoet exhibits generalization capability
in processing new text, image, or video inputs that diverge
from the training data distribution. Furthermore, VideoPoet
begins to show an ability to handle new tasks that were not
included in its training. For example, VideoPoet demon-
strates the ability to perform new editing tasks by sequen-
tially chaining training tasks together, as elaborated in Sec-
tion 7.
VideoPoet