title: Taming Teacher Forcing for Masked Autoregressive Video Generation
summary: We introduce MAGI, a hybrid video generation framework that combines masked
  modeling for intra-frame generation with causal modeling for next-frame
  generation. Our key innovation, Complete Teacher Forcing (CTF), conditions
  masked frames on complete observation frames rather than masked ones (namely
  Masked Teacher Forcing, MTF), enabling a smooth transition from token-level
  (patch-level) to frame-level autoregressive generation. CTF significantly
  outperforms MTF, achieving a +23% improvement in FVD scores on first-frame
  conditioned video prediction. To address issues like exposure bias, we employ
  targeted training strategies, setting a new benchmark in autoregressive video
  generation. Experiments show that MAGI can generate long, coherent video
  sequences exceeding 100 frames, even when trained on as few as 16 frames,
  highlighting its potential for scalable, high-quality video generation.
paper_authors._id: 6791564a65b96bef4a168e00
paper_authors.name: Deyu Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T20:54:25.055Z
paper_authors._id: 64e821f2bddc5b1072b15c2e
paper_authors.avatarUrl: /avatars/618b5a48f2fa62daff4e1922a9aa9e8b.svg
paper_authors.fullname: zhoudeyu
paper_authors.type: user
paper_authors.user: zhoudeyu
paper_authors._id: 6791564a65b96bef4a168e01
paper_authors.name: Quan Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T20:54:39.129Z
paper_authors._id: 630d7a8f81ef9b1772b67f4c
paper_authors.avatarUrl: /avatars/00757abd6e548ccebb5bfb233be129a2.svg
paper_authors.fullname: Quan Sun
paper_authors.type: user
paper_authors.user: QuanSun
paper_authors._id: 6791564a65b96bef4a168e02
paper_authors.name: Yuang Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T20:54:44.656Z
paper_authors._id: 631ee086c1a8269da39265c6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631ee086c1a8269da39265c6/wUa1epGtTGcUv2mvrLUcD.png
paper_authors.fullname: Yuang Peng
paper_authors.type: user
paper_authors.user: yuangpeng
paper_authors._id: 6791564a65b96bef4a168e03
paper_authors.name: Kun Yan
paper_authors._id: 6791564a65b96bef4a168e04
paper_authors.name: Runpei Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-22T20:49:49.933Z
paper_authors._id: 6201fc5d91d53938a6432fbf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6201fc5d91d53938a6432fbf/VLs8ZYaZrop4KBpZn53fH.jpeg
paper_authors.fullname: Runpei Dong
paper_authors.type: user
paper_authors.user: RunpeiDong
paper_authors._id: 6791564a65b96bef4a168e05
paper_authors.name: Duomin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-23T09:18:05.586Z
paper_authors._id: 64ae9b88a22a179fc4d07992
paper_authors.avatarUrl: /avatars/c9065f04a1188ea3129e56a90328ffd3.svg
paper_authors.fullname: wang
paper_authors.type: user
paper_authors.user: dorni
paper_authors._id: 6791564a65b96bef4a168e06
paper_authors.name: Zheng Ge
paper_authors._id: 6791564a65b96bef4a168e07
paper_authors.name: Nan Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T20:55:16.663Z
paper_authors._id: 66422aede3e9eba09d3cb753
paper_authors.avatarUrl: /avatars/52a6292fcc3a37265062bdcfbea73441.svg
paper_authors.fullname: Nan Duan
paper_authors.type: user
paper_authors.user: opotle
paper_authors._id: 6791564a65b96bef4a168e08
paper_authors.name: Xiangyu Zhang
paper_authors._id: 6791564a65b96bef4a168e09
paper_authors.name: Lionel M. Ni
paper_authors._id: 6791564a65b96bef4a168e0a
paper_authors.name: Heung-Yeung Shum
paper_title: Taming Teacher Forcing for Masked Autoregressive Video Generation
paper_summary: We introduce MAGI, a hybrid video generation framework that combines masked
  modeling for intra-frame generation with causal modeling for next-frame
  generation. Our key innovation, Complete Teacher Forcing (CTF), conditions
  masked frames on complete observation frames rather than masked ones (namely
  Masked Teacher Forcing, MTF), enabling a smooth transition from token-level
  (patch-level) to frame-level autoregressive generation. CTF significantly
  outperforms MTF, achieving a +23% improvement in FVD scores on first-frame
  conditioned video prediction. To address issues like exposure bias, we employ
  targeted training strategies, setting a new benchmark in autoregressive video
  generation. Experiments show that MAGI can generate long, coherent video
  sequences exceeding 100 frames, even when trained on as few as 16 frames,
  highlighting its potential for scalable, high-quality video generation.
paper_ai_keywords: MAGI dense video generation masked modeling intra-frame generation causal modeling next-frame generation Complete Teacher Forcing (CTF) Masked Teacher Forcing (MTF) token-level patch-level frame-level autoregressive generation FVD scores first-frame conditioned video prediction exposure bias targeted training strategies long, coherent video generation scalable, high-quality video generation