title: Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
summary: Sora is the first large-scale generalist video generation model that garnered
  significant attention across society. Since its launch by OpenAI in February
  2024, no other video generation models have paralleled {Sora}'s performance or
  its capacity to support a broad spectrum of video generation tasks.
  Additionally, there are only a few fully published video generation models,
  with the majority being closed-source. To address this gap, this paper proposes
  a new multi-agent framework Mora, which incorporates several advanced visual AI
  agents to replicate generalist video generation demonstrated by Sora. In
  particular, Mora can utilize multiple visual agents and successfully mimic
  Sora's video generation capabilities in various tasks, such as (1)
  text-to-video generation, (2) text-conditional image-to-video generation, (3)
  extend generated videos, (4) video-to-video editing, (5) connect videos and (6)
  simulate digital worlds. Our extensive experimental results show that Mora
  achieves performance that is proximate to that of Sora in various tasks.
  However, there exists an obvious performance gap between our work and Sora when
  assessed holistically. In summary, we hope this project can guide the future
  trajectory of video generation through collaborative AI agents.
paper_authors._id: 65fb8f07a12d7b72305fab75
paper_authors.name: Zhengqing Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T09:00:27.646Z
paper_authors._id: 62f3cc40b3e6016d46027e1d
paper_authors.avatarUrl: /avatars/9b26d2fd5e8574f633ffd97af1700b45.svg
paper_authors.fullname: YuanZhengqing
paper_authors.type: user
paper_authors.user: YuanZhengqing
paper_authors._id: 65fb8f07a12d7b72305fab76
paper_authors.name: Ruoxi Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T09:00:34.616Z
paper_authors._id: 65a135df18b7d7c225737173
paper_authors.avatarUrl: /avatars/a186f776011bb62c54fd73e787ff8415.svg
paper_authors.fullname: Ruoxi Chen
paper_authors.type: user
paper_authors.user: Dipsy0830
paper_authors._id: 65fb8f07a12d7b72305fab77
paper_authors.name: Zhaoxu Li
paper_authors._id: 65fb8f07a12d7b72305fab78
paper_authors.name: Haolong Jia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-21T08:26:52.790Z
paper_authors._id: 636af1370d7b46535de59ddf
paper_authors.avatarUrl: /avatars/f57b5a074491ce20611f4e08591b8815.svg
paper_authors.fullname: Haolong Jia
paper_authors.type: user
paper_authors.user: JHL328
paper_authors._id: 65fb8f07a12d7b72305fab79
paper_authors.name: Lifang He
paper_authors._id: 65fb8f07a12d7b72305fab7a
paper_authors.name: Chi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T09:01:36.725Z
paper_authors._id: 640baf807047b6c332a912e1
paper_authors.avatarUrl: /avatars/830fd35aff114eedf68c4f9eeb9fe944.svg
paper_authors.fullname: Chi Wang
paper_authors.type: user
paper_authors.user: sonichi
paper_authors._id: 65fb8f07a12d7b72305fab7b
paper_authors.name: Lichao Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-21T08:26:50.195Z
paper_authors._id: 65a52766215aabac489e3468
paper_authors.avatarUrl: /avatars/fe05e22cd7e12e961296426434e17c76.svg
paper_authors.fullname: Lichao Sun
paper_authors.type: user
paper_authors.user: sunlichao137
paper_title: Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
paper_summary: Sora is the first large-scale generalist video generation model that garnered
  significant attention across society. Since its launch by OpenAI in February
  2024, no other video generation models have paralleled {Sora}'s performance or
  its capacity to support a broad spectrum of video generation tasks.
  Additionally, there are only a few fully published video generation models,
  with the majority being closed-source. To address this gap, this paper proposes
  a new multi-agent framework Mora, which incorporates several advanced visual AI
  agents to replicate generalist video generation demonstrated by Sora. In
  particular, Mora can utilize multiple visual agents and successfully mimic
  Sora's video generation capabilities in various tasks, such as (1)
  text-to-video generation, (2) text-conditional image-to-video generation, (3)
  extend generated videos, (4) video-to-video editing, (5) connect videos and (6)
  simulate digital worlds. Our extensive experimental results show that Mora
  achieves performance that is proximate to that of Sora in various tasks.
  However, there exists an obvious performance gap between our work and Sora when
  assessed holistically. In summary, we hope this project can guide the future
  trajectory of video generation through collaborative AI agents.
paper_ai_keywords: video generation model multi-agent framework visual AI agents text-to-video generation text-conditional image-to-video generation video-to-video editing simulate digital worlds