Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
Sora is the first large-scale generalist video generation model that garnered
significant attention across society. Since its launch by OpenAI in February
2024, no other video generation models have paralleled {Sora}'s performance or
its capacity to support a broad spectrum of video generation tasks.
Additionally, there are only a few fully published video generation models,
with the majority being closed-source. To address this gap, this paper proposes
a new multi-agent framework Mora, which incorporates several advanced visual AI
agents to replicate generalist video generation demonstrated by Sora. In
particular, Mora can utilize multiple visual agents and successfully mimic
Sora's video generation capabilities in various tasks, such as (1)
text-to-video generation, (2) text-conditional image-to-video generation, (3)
extend generated videos, (4) video-to-video editing, (5) connect videos and (6)
simulate digital worlds. Our extensive experimental results show that Mora
achieves performance that is proximate to that of Sora in various tasks.
However, there exists an obvious performance gap between our work and Sora when
assessed holistically. In summary, we hope this project can guide the future
trajectory of video generation through collaborative AI agents.
65fb8f07a12d7b72305fab75
Zhengqing Yuan
admin_assigned
2024-03-21T09:00:27.646Z
62f3cc40b3e6016d46027e1d
/avatars/9b26d2fd5e8574f633ffd97af1700b45.svg
YuanZhengqing
user
YuanZhengqing
65fb8f07a12d7b72305fab76
Ruoxi Chen
admin_assigned
2024-03-21T09:00:34.616Z
65a135df18b7d7c225737173
/avatars/cc829bf663da9ca9df9d92feea80e42f.svg
Ruoxi Chen
user
Dipsy0830
65fb8f07a12d7b72305fab77
Zhaoxu Li
65fb8f07a12d7b72305fab78
Haolong Jia
claimed_verified
2024-03-21T08:26:52.790Z
636af1370d7b46535de59ddf
/avatars/f57b5a074491ce20611f4e08591b8815.svg
Haolong Jia
user
JHL328
65fb8f07a12d7b72305fab79
Lifang He
65fb8f07a12d7b72305fab7a
Chi Wang
admin_assigned
2024-03-21T09:01:36.725Z
640baf807047b6c332a912e1
/avatars/830fd35aff114eedf68c4f9eeb9fe944.svg
Chi Wang
user
sonichi
65fb8f07a12d7b72305fab7b
Lichao Sun
claimed_verified
2024-03-21T08:26:50.195Z
65a52766215aabac489e3468
/avatars/fe05e22cd7e12e961296426434e17c76.svg
Lichao Sun
user
sunlichao137
Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
Sora is the first large-scale generalist video generation model that garnered
significant attention across society. Since its launch by OpenAI in February
2024, no other video generation models have paralleled {Sora}'s performance or
its capacity to support a broad spectrum of video generation tasks.
Additionally, there are only a few fully published video generation models,
with the majority being closed-source. To address this gap, this paper proposes
a new multi-agent framework Mora, which incorporates several advanced visual AI
agents to replicate generalist video generation demonstrated by Sora. In
particular, Mora can utilize multiple visual agents and successfully mimic
Sora's video generation capabilities in various tasks, such as (1)
text-to-video generation, (2) text-conditional image-to-video generation, (3)
extend generated videos, (4) video-to-video editing, (5) connect videos and (6)
simulate digital worlds. Our extensive experimental results show that Mora
achieves performance that is proximate to that of Sora in various tasks.
However, there exists an obvious performance gap between our work and Sora when
assessed holistically. In summary, we hope this project can guide the future
trajectory of video generation through collaborative AI agents.
A new multi-agent framework Mora replicates Sora's generalist video generation capabilities using several advanced visual AI agents, achieving similar performance across various tasks.
text-to-video generation text-conditional image-to-video generation extend generated videos video-to-video editing connect videos simulate digital worlds multi-agent framework visual AI agents