Figure 1. Pipeline comparison of our GPT4Video with the current
LLMs and MLLMs.

remarkable zero-shot capabilities across diverse linguis-
tic tasks. Based on this foundation, multimodal large
language models (MLLMs) like MiniGPT-4 [59], Video-
LLaMA [52], Macaw-LLM [27] (as shown in Figure 1 (b))
have been successively introduced. These models are capa-
ble not only of processing textual information but also of
handling data across various modalities such as images, au-
dio, and videos, demonstrating exceptional performance in
multimodal tasks [51]. Nevertheless, current MLLMs pri-
marily focus on processing multimodal inputs, yet they fall
short in generating multimodal information on the output
side. For a refined AGI system [13], in addition to the ca-
pability of multimodal understanding, it is equally pivotal
to possess an efficient multimodal generation capacity, as
this aligns more closely with human interaction modalities
in real-world contexts.
GPT4Video