title: Movie Gen: A Cast of Media Foundation Models
summary: We present Movie Gen, a cast of foundation models that generates
  high-quality, 1080p HD videos with different aspect ratios and synchronized
  audio. We also show additional capabilities such as precise instruction-based
  video editing and generation of personalized videos based on a user's image.
  Our models set a new state-of-the-art on multiple tasks: text-to-video
  synthesis, video personalization, video editing, video-to-audio generation, and
  text-to-audio generation. Our largest video generation model is a 30B parameter
  transformer trained with a maximum context length of 73K video tokens,
  corresponding to a generated video of 16 seconds at 16 frames-per-second. We
  show multiple technical innovations and simplifications on the architecture,
  latent spaces, training objectives and recipes, data curation, evaluation
  protocols, parallelization techniques, and inference optimizations that allow
  us to reap the benefits of scaling pre-training data, model size, and training
  compute for training large scale media generation models. We hope this paper
  helps the research community to accelerate progress and innovation in media
  generation models. All videos from this paper are available at
  https://go.fb.me/MovieGenResearchVideos.
paper_authors._id: 6711d977b265bbd413b8320c
paper_authors.name: Adam Polyak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:16:24.972Z
paper_authors._id: 6304d514dae2eb7d08413d62
paper_authors.avatarUrl: /avatars/02a571bc791b78d3993d9a0484b70a29.svg
paper_authors.fullname: Adam Polyak
paper_authors.type: user
paper_authors.user: adampo
paper_authors._id: 6711d977b265bbd413b8320d
paper_authors.name: Amit Zohar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:16:30.835Z
paper_authors._id: 64d8cb34505306fcd2fb89c3
paper_authors.avatarUrl: /avatars/4ae9ffe5d494a7d53bc519ff8e402b3e.svg
paper_authors.fullname: Amit Zohar
paper_authors.type: user
paper_authors.user: amitz
paper_authors._id: 6711d977b265bbd413b8320e
paper_authors.name: Andrew Brown
paper_authors._id: 6711d977b265bbd413b8320f
paper_authors.name: Andros Tjandra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:16:39.991Z
paper_authors._id: 647779c6c21496284db45835
paper_authors.avatarUrl: /avatars/c1176e57f5d0aaa5f50bbde48a19af01.svg
paper_authors.fullname: Andros Tjandra
paper_authors.type: user
paper_authors.user: androstj
paper_authors._id: 6711d977b265bbd413b83210
paper_authors.name: Animesh Sinha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:17:19.207Z
paper_authors._id: 63772d28e069bdcdbb8e4a29
paper_authors.avatarUrl: /avatars/be71b00e009f2093a3ff79f07815e68e.svg
paper_authors.fullname: Animesh Sinha
paper_authors.type: user
paper_authors.user: animeshsinha
paper_authors._id: 6711d977b265bbd413b83211
paper_authors.name: Ann Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:17:31.419Z
paper_authors._id: 663353017263322326bb044e
paper_authors.avatarUrl: /avatars/ac6a5b814592e4c3cd84d7255e31ec88.svg
paper_authors.fullname: Yi-hsuan Lee
paper_authors.type: user
paper_authors.user: AnnLee
paper_authors._id: 6711d977b265bbd413b83212
paper_authors.name: Apoorv Vyas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:17:01.338Z
paper_authors._id: 638697308b5acae8d251cdec
paper_authors.avatarUrl: /avatars/dcfc47ee012439cc5ecc28e48ff26983.svg
paper_authors.fullname: Apoorv Vyas
paper_authors.type: user
paper_authors.user: apoorv2904
paper_authors._id: 6711d977b265bbd413b83213
paper_authors.name: Bowen Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:17:46.422Z
paper_authors._id: 6477d6e06e6c7ac608c44734
paper_authors.avatarUrl: /avatars/c6d126c5b2930531ac4b00a1ad5e6d26.svg
paper_authors.fullname: Bowen Shi
paper_authors.type: user
paper_authors.user: bowenshi
paper_authors._id: 6711d977b265bbd413b83214
paper_authors.name: Chih-Yao Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:18:03.265Z
paper_authors._id: 6515bf00c2aedaa7e692a1ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6515bf00c2aedaa7e692a1ec/PYpS1QwTPdQYW5_BrvwzP.png
paper_authors.fullname: Chih-Yao Ma
paper_authors.type: user
paper_authors.user: chihyaoma
paper_authors._id: 6711d977b265bbd413b83215
paper_authors.name: Ching-Yao Chuang
paper_authors._id: 6711d977b265bbd413b83216
paper_authors.name: David Yan
paper_authors._id: 6711d977b265bbd413b83217
paper_authors.name: Dhruv Choudhary
paper_authors._id: 6711d977b265bbd413b83218
paper_authors.name: Dingkang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:19:33.523Z
paper_authors._id: 65dbd00d730386b492a88c06
paper_authors.avatarUrl: /avatars/362ae2bf1144d0bbb822fad3c33f239c.svg
paper_authors.fullname: Dingkang Wang
paper_authors.type: user
paper_authors.user: wangdk93
paper_authors._id: 6711d977b265bbd413b83219
paper_authors.name: Geet Sethi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:19:43.218Z
paper_authors._id: 653405a002d1ecd5455d4f0a
paper_authors.avatarUrl: /avatars/fc5ab33a508dadec6d2e15e992e16dec.svg
paper_authors.fullname: Geet Sethi
paper_authors.type: user
paper_authors.user: geetsethi
paper_authors._id: 6711d977b265bbd413b8321a
paper_authors.name: Guan Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:19:51.272Z
paper_authors._id: 6573a458d7f487de5f9bb44a
paper_authors.avatarUrl: /avatars/4aa5626af43855ad770186f0d4191a01.svg
paper_authors.fullname: Guan Pang
paper_authors.type: user
paper_authors.user: tbpangolin
paper_authors._id: 6711d977b265bbd413b8321b
paper_authors.name: Haoyu Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:20:03.497Z
paper_authors._id: 650a8979c19e5b4c8a6ff062
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650a8979c19e5b4c8a6ff062/64_JuECX_k_-uK7m7nlua.jpeg
paper_authors.fullname: Haoyu Ma
paper_authors.type: user
paper_authors.user: haoyum1997
paper_authors._id: 6711d977b265bbd413b8321c
paper_authors.name: Ishan Misra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:20:11.699Z
paper_authors._id: 63faa5b04380ab0cb9577a57
paper_authors.avatarUrl: /avatars/c71be0acb0553671c9703654e998cbd5.svg
paper_authors.fullname: Ishan Misra
paper_authors.type: user
paper_authors.user: hayhay404
paper_authors._id: 6711d977b265bbd413b8321d
paper_authors.name: Ji Hou
paper_authors._id: 6711d977b265bbd413b8321e
paper_authors.name: Jialiang Wang
paper_authors._id: 6711d977b265bbd413b8321f
paper_authors.name: Kiran Jagadeesh
paper_authors._id: 6711d977b265bbd413b83220
paper_authors.name: Kunpeng Li
paper_authors._id: 6711d977b265bbd413b83221
paper_authors.name: Luxin Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:21:46.568Z
paper_authors._id: 65a4fa7d2548c41ad9d9b710
paper_authors.avatarUrl: /avatars/3cace2d2f11f7194d8eca4b95b0b57cc.svg
paper_authors.fullname: Luxin Zhang
paper_authors.type: user
paper_authors.user: Luczzz
paper_authors._id: 6711d977b265bbd413b83222
paper_authors.name: Mannat Singh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:22:01.617Z
paper_authors._id: 650fece35877b1c077298b81
paper_authors.avatarUrl: /avatars/be9c948ecf9c413ad4c8aa6598d3ba86.svg
paper_authors.fullname: Mannat Singh
paper_authors.type: user
paper_authors.user: mannatsingh
paper_authors._id: 6711d977b265bbd413b83223
paper_authors.name: Mary Williamson
paper_authors._id: 6711d977b265bbd413b83224
paper_authors.name: Matt Le
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T14:16:58.580Z
paper_authors._id: 6311fba9fdb55de45d2245ea
paper_authors.avatarUrl: /avatars/5799e56b737d78550bf8d164cbe3d41c.svg
paper_authors.fullname: Matt Le
paper_authors.type: user
paper_authors.user: lematt1991
paper_authors._id: 6711d977b265bbd413b83225
paper_authors.name: Matthew Yu
paper_authors._id: 6711d977b265bbd413b83226
paper_authors.name: Mitesh Kumar Singh
paper_authors._id: 6711d977b265bbd413b83227
paper_authors.name: Peizhao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:22:59.119Z
paper_authors._id: 6433a635a5aed21dd110381e
paper_authors.avatarUrl: /avatars/b1fa4fe1c8bd7989875233820fa425af.svg
paper_authors.fullname: Peizhao Zhang
paper_authors.type: user
paper_authors.user: stzpz
paper_authors._id: 6711d977b265bbd413b83228
paper_authors.name: Peter Vajda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:23:12.630Z
paper_authors._id: 636f4b35af79dd193603342b
paper_authors.avatarUrl: /avatars/577328195738d92ac41a3d0ff3108d7c.svg
paper_authors.fullname: Peter Vajda
paper_authors.type: user
paper_authors.user: vaphab
paper_authors._id: 6711d977b265bbd413b83229
paper_authors.name: Quentin Duval
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:23:18.989Z
paper_authors._id: 66a1632f40c32a6a39ba0c51
paper_authors.avatarUrl: /avatars/5f577e016fa68e65b72fb1f154c88288.svg
paper_authors.fullname: Quentin Duval
paper_authors.type: user
paper_authors.user: qduval
paper_authors._id: 6711d977b265bbd413b8322a
paper_authors.name: Rohit Girdhar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:23:27.698Z
paper_authors._id: 63c6c737656e7822e23732d8
paper_authors.avatarUrl: /avatars/3cf062864dbf2b5423eb1f4beaeec1ea.svg
paper_authors.fullname: Rohit Girdhar
paper_authors.type: user
paper_authors.user: rgirdhar
paper_authors._id: 6711d977b265bbd413b8322b
paper_authors.name: Roshan Sumbaly
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:23:34.133Z
paper_authors._id: 6515c595f464644a73b50064
paper_authors.avatarUrl: /avatars/119102c41fb8a7986c72b3f55ec6200f.svg
paper_authors.fullname: Roshan sumbaly
paper_authors.type: user
paper_authors.user: rsumbaly
paper_authors._id: 6711d977b265bbd413b8322c
paper_authors.name: Sai Saketh Rambhatla
paper_authors._id: 6711d977b265bbd413b8322d
paper_authors.name: Sam Tsai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T15:31:35.305Z
paper_authors._id: 67127b753a058517519fabe8
paper_authors.avatarUrl: /avatars/6fabb1634efb8ad8a8a2ae96fd6cf1c6.svg
paper_authors.fullname: Sam Tsai
paper_authors.type: user
paper_authors.user: sstsai-adl
paper_authors._id: 6711d977b265bbd413b8322e
paper_authors.name: Samaneh Azadi
paper_authors._id: 6711d977b265bbd413b8322f
paper_authors.name: Samyak Datta
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:24:02.672Z
paper_authors._id: 657c9c9242fc53e18b00a3bc
paper_authors.avatarUrl: /avatars/0365d677a8151632f05eff140d8cdb62.svg
paper_authors.fullname: Samyak Datta
paper_authors.type: user
paper_authors.user: samyakdatta
paper_authors._id: 6711d977b265bbd413b83230
paper_authors.name: Sanyuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:24:10.750Z
paper_authors._id: 6465843ce27766e8921b9cab
paper_authors.avatarUrl: /avatars/6017db24da593c9afb21edfcedcddfd4.svg
paper_authors.fullname: Sanyuan Chen
paper_authors.type: user
paper_authors.user: Sanyuan-Chen
paper_authors._id: 6711d977b265bbd413b83231
paper_authors.name: Sean Bell
paper_authors._id: 6711d977b265bbd413b83232
paper_authors.name: Sharadh Ramaswamy
paper_authors._id: 6711d977b265bbd413b83233
paper_authors.name: Shelly Sheynin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:24:59.545Z
paper_authors._id: 63468e6ca6f101c7f9049132
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665662111525-63468e6ca6f101c7f9049132.png
paper_authors.fullname: Shelly Sheynin
paper_authors.type: user
paper_authors.user: shellysheynin
paper_authors._id: 6711d977b265bbd413b83234
paper_authors.name: Siddharth Bhattacharya
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:24:19.232Z
paper_authors._id: 65d3c3fe974775cbcbf3397a
paper_authors.avatarUrl: /avatars/06c1a26cee0ffad7d318a24c4a16ad9d.svg
paper_authors.fullname: Siddharth Bhattacharya
paper_authors.type: user
paper_authors.user: Siddharth96
paper_authors._id: 6711d977b265bbd413b83235
paper_authors.name: Simran Motwani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:25:07.110Z
paper_authors._id: 6515bb7e99e8060f1e792047
paper_authors.avatarUrl: /avatars/9c6400f83c3165de3154b0e6bee2e5a1.svg
paper_authors.fullname: Simran Motwani
paper_authors.type: user
paper_authors.user: smotwani
paper_authors._id: 6711d977b265bbd413b83236
paper_authors.name: Tao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:25:15.774Z
paper_authors._id: 632bc84fca316c73cd40ca5c
paper_authors.avatarUrl: /avatars/5b000f86f8161f7a4bc4200e4b76a6e1.svg
paper_authors.fullname: Tao Xu
paper_authors.type: user
paper_authors.user: taoxu
paper_authors._id: 6711d977b265bbd413b83237
paper_authors.name: Tianhe Li
paper_authors._id: 6711d977b265bbd413b83238
paper_authors.name: Tingbo Hou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:25:44.779Z
paper_authors._id: 655846d7ed8df83128f5826a
paper_authors.avatarUrl: /avatars/d7ce174d7d1b8614d5f6f071225c0057.svg
paper_authors.fullname: Hou
paper_authors.type: user
paper_authors.user: Tingbo
paper_authors._id: 6711d977b265bbd413b83239
paper_authors.name: Wei-Ning Hsu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:25:56.603Z
paper_authors._id: 64c143d484887aa2b43bac0d
paper_authors.avatarUrl: /avatars/817dba24739676aa0f34692702bb00d6.svg
paper_authors.fullname: Wei-Ning Hsu
paper_authors.type: user
paper_authors.user: wnhsu
paper_authors._id: 6711d977b265bbd413b8323a
paper_authors.name: Xi Yin
paper_authors._id: 6711d977b265bbd413b8323b
paper_authors.name: Xiaoliang Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:26:10.362Z
paper_authors._id: 6549417b3ce45eb764faf993
paper_authors.avatarUrl: /avatars/d310f475d0697f5f13b3d4141ea0ccaf.svg
paper_authors.fullname: Xiaoliang Dai
paper_authors.type: user
paper_authors.user: daixl1992
paper_authors._id: 6711d977b265bbd413b8323c
paper_authors.name: Yaniv Taigman
paper_authors._id: 6711d977b265bbd413b8323d
paper_authors.name: Yaqiao Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:26:55.559Z
paper_authors._id: 63d2a5a6c20657d24347000c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674749322268-noauth.jpeg
paper_authors.fullname: Luo
paper_authors.type: user
paper_authors.user: Yaqiao
paper_authors._id: 6711d977b265bbd413b8323e
paper_authors.name: Yen-Cheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:27:11.636Z
paper_authors._id: 620607bc980cf06cf5b1cd3b
paper_authors.avatarUrl: /avatars/3a49b4bd068ca4ec9e16edd65862ce5b.svg
paper_authors.fullname: Yen-Cheng Liu
paper_authors.type: user
paper_authors.user: ycliu
paper_authors._id: 6711d977b265bbd413b8323f
paper_authors.name: Yi-Chiao Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:27:04.005Z
paper_authors._id: 6410ebe8028e6f7153df0842
paper_authors.avatarUrl: /avatars/96960d2c7d713510d25903fdabe5b825.svg
paper_authors.fullname: Yi-Chiao Wu
paper_authors.type: user
paper_authors.user: yichiaowu
paper_authors._id: 6711d977b265bbd413b83240
paper_authors.name: Yue Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:31:09.407Z
paper_authors._id: 641cc612f0b71a9743668604
paper_authors.avatarUrl: /avatars/ab7d91df8de266fadb4e5632a1f0a37a.svg
paper_authors.fullname: Yue Zhao
paper_authors.type: user
paper_authors.user: yzhao30
paper_authors._id: 6711d977b265bbd413b83241
paper_authors.name: Yuval Kirstain
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:27:22.706Z
paper_authors._id: 6551fa1b5491c57ee900108b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6551fa1b5491c57ee900108b/HXEfGUJRExQpfGpX3F8N0.png
paper_authors.fullname: Yuval Kirstain
paper_authors.type: user
paper_authors.user: yuvalkirstainmeta
paper_authors._id: 6711d977b265bbd413b83242
paper_authors.name: Zecheng He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:27:38.060Z
paper_authors._id: 641b7d42ec5b871c0bcf2955
paper_authors.avatarUrl: /avatars/f6e24742b4befa889e72c7e5eefd6cf6.svg
paper_authors.fullname: He
paper_authors.type: user
paper_authors.user: zechengh
paper_authors._id: 6711d977b265bbd413b83243
paper_authors.name: Zijian He
paper_authors._id: 6711d977b265bbd413b83244
paper_authors.name: Albert Pumarola
paper_authors._id: 6711d977b265bbd413b83245
paper_authors.name: Ali Thabet
paper_authors._id: 6711d977b265bbd413b83246
paper_authors.name: Artsiom Sanakoyeu
paper_authors._id: 6711d977b265bbd413b83247
paper_authors.name: Arun Mallya
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:28:09.467Z
paper_authors._id: 6513374d2f79557a90c0f0fe
paper_authors.avatarUrl: /avatars/96eadc6f0e0f91ea3d874df207a9db71.svg
paper_authors.fullname: Arun Mallya
paper_authors.type: user
paper_authors.user: ammp
paper_authors._id: 6711d977b265bbd413b83248
paper_authors.name: Baishan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:28:32.745Z
paper_authors._id: 66a8050feb641bdde6e60b6b
paper_authors.avatarUrl: /avatars/b239d0eeaecda333a04d15fdd0a106c7.svg
paper_authors.fullname: Baishan Guo
paper_authors.type: user
paper_authors.user: edwardguo
paper_authors._id: 6711d977b265bbd413b83249
paper_authors.name: Boris Araya
paper_authors._id: 6711d977b265bbd413b8324a
paper_authors.name: Breena Kerr
paper_authors._id: 6711d977b265bbd413b8324b
paper_authors.name: Carleigh Wood
paper_authors._id: 6711d977b265bbd413b8324c
paper_authors.name: Ce Liu
paper_authors._id: 6711d977b265bbd413b8324d
paper_authors.name: Cen Peng
paper_authors._id: 6711d977b265bbd413b8324e
paper_authors.name: Dimitry Vengertsev
paper_authors._id: 6711d977b265bbd413b8324f
paper_authors.name: Edgar Schonfeld
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T14:16:57.158Z
paper_authors._id: 6571c662d5c6a6d3b0bdae88
paper_authors.avatarUrl: /avatars/8080bcdf8b331f62383b724050189660.svg
paper_authors.fullname: Edgar Schoenfeld
paper_authors.type: user
paper_authors.user: edgarschoenfeld
paper_authors._id: 6711d977b265bbd413b83250
paper_authors.name: Elliot Blanchard
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:30:11.640Z
paper_authors._id: 6563cf4a718b9db5215d07b4
paper_authors.avatarUrl: /avatars/20e5be0bd3fcd888959a0af0f0b462e4.svg
paper_authors.fullname: Elliot Blanchard
paper_authors.type: user
paper_authors.user: elliotblanchard
paper_authors._id: 6711d977b265bbd413b83251
paper_authors.name: Felix Juefei-Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-16T16:55:38.589Z
paper_authors._id: 6417cf37dce1e4c0229f17b1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6417cf37dce1e4c0229f17b1/7h-ZCB5f4wif7TsnF-B1M.jpeg
paper_authors.fullname: Felix Xu
paper_authors.type: user
paper_authors.user: katanaxu
paper_authors._id: 6711d977b265bbd413b83252
paper_authors.name: Fraylie Nord
paper_authors._id: 6711d977b265bbd413b83253
paper_authors.name: Jeff Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:30:33.700Z
paper_authors._id: 63631be58f43a912fc7595d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1667440137801-63631be58f43a912fc7595d5.jpeg
paper_authors.fullname: Jeff Liang
paper_authors.type: user
paper_authors.user: JeffLiang
paper_authors._id: 6711d977b265bbd413b83254
paper_authors.name: John Hoffman
paper_authors._id: 6711d977b265bbd413b83255
paper_authors.name: Jonas Kohler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:30:01.260Z
paper_authors._id: 6408a3a19e9f790c905281c2
paper_authors.avatarUrl: /avatars/3517962e54bda141018e13f7e21fb1ae.svg
paper_authors.fullname: jonas köhler
paper_authors.type: user
paper_authors.user: Jonaskohler
paper_authors._id: 6711d977b265bbd413b83256
paper_authors.name: Kaolin Fire
paper_authors._id: 6711d977b265bbd413b83257
paper_authors.name: Karthik Sivakumar
paper_authors._id: 6711d977b265bbd413b83258
paper_authors.name: Lawrence Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:31:47.867Z
paper_authors._id: 6303ff0c7b50dd9d0a3777e8
paper_authors.avatarUrl: /avatars/1c65d5936fe0cafd1fadd1102fad28db.svg
paper_authors.fullname: Lawrence Chen
paper_authors.type: user
paper_authors.user: lawrencechen
paper_authors._id: 6711d977b265bbd413b83259
paper_authors.name: Licheng Yu
paper_authors._id: 6711d977b265bbd413b8325a
paper_authors.name: Luya Gao
paper_authors._id: 6711d977b265bbd413b8325b
paper_authors.name: Markos Georgopoulos
paper_authors._id: 6711d977b265bbd413b8325c
paper_authors.name: Rashel Moritz
paper_authors._id: 6711d977b265bbd413b8325d
paper_authors.name: Sara K. Sampson
paper_authors._id: 6711d977b265bbd413b8325e
paper_authors.name: Shikai Li
paper_authors._id: 6711d977b265bbd413b8325f
paper_authors.name: Simone Parmeggiani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:32:42.813Z
paper_authors._id: 66a046fe7bcf5121db35e4a6
paper_authors.avatarUrl: /avatars/4d407ab2245f63610c41bba602f1c0f8.svg
paper_authors.fullname: Simone Parmeggiani
paper_authors.type: user
paper_authors.user: ngb81
paper_authors._id: 6711d977b265bbd413b83260
paper_authors.name: Steve Fine
paper_authors._id: 6711d977b265bbd413b83261
paper_authors.name: Tara Fowler
paper_authors._id: 6711d977b265bbd413b83262
paper_authors.name: Vladan Petrovic
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:24:45.907Z
paper_authors._id: 6515caf71e7b9224c9d23851
paper_authors.avatarUrl: /avatars/11b17ba6d4f8e7b97a1644cc551a9fe7.svg
paper_authors.fullname: Vladan Petrovic
paper_authors.type: user
paper_authors.user: vpetrovi
paper_authors._id: 6711d977b265bbd413b83263
paper_authors.name: Yuming Du
paper_title: Movie Gen: A Cast of Media Foundation Models
paper_summary: We present Movie Gen, a cast of foundation models that generates
  high-quality, 1080p HD videos with different aspect ratios and synchronized
  audio. We also show additional capabilities such as precise instruction-based
  video editing and generation of personalized videos based on a user's image.
  Our models set a new state-of-the-art on multiple tasks: text-to-video
  synthesis, video personalization, video editing, video-to-audio generation, and
  text-to-audio generation. Our largest video generation model is a 30B parameter
  transformer trained with a maximum context length of 73K video tokens,
  corresponding to a generated video of 16 seconds at 16 frames-per-second. We
  show multiple technical innovations and simplifications on the architecture,
  latent spaces, training objectives and recipes, data curation, evaluation
  protocols, parallelization techniques, and inference optimizations that allow
  us to reap the benefits of scaling pre-training data, model size, and training
  compute for training large scale media generation models. We hope this paper
  helps the research community to accelerate progress and innovation in media
  generation models. All videos from this paper are available at
  https://go.fb.me/MovieGenResearchVideos.
paper_ai_keywords: foundation models text-to-video synthesis video personalization video editing video-to-audio generation text-to-audio generation transformer video tokens latent spaces training objectives data curation evaluation protocols parallelization techniques inference optimizations