title: History-Guided Video Diffusion
summary: Classifier-free guidance (CFG) is a key technique for improving conditional
  generation in diffusion models, enabling more accurate control while enhancing
  sample quality. It is natural to extend this technique to video diffusion,
  which generates video conditioned on a variable number of context frames,
  collectively referred to as history. However, we find two key challenges to
  guiding with variable-length history: architectures that only support
  fixed-size conditioning, and the empirical observation that CFG-style history
  dropout performs poorly. To address this, we propose the Diffusion Forcing
  Transformer (DFoT), a video diffusion architecture and theoretically grounded
  training objective that jointly enable conditioning on a flexible number of
  history frames. We then introduce History Guidance, a family of guidance
  methods uniquely enabled by DFoT. We show that its simplest form, vanilla
  history guidance, already significantly improves video generation quality and
  temporal consistency. A more advanced method, history guidance across time and
  frequency further enhances motion dynamics, enables compositional
  generalization to out-of-distribution history, and can stably roll out
  extremely long videos. Website: https://boyuan.space/history-guidance
paper_authors._id: 67aac6052c02e43558b6b4b0
paper_authors.name: Kiwhan Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-12T09:16:57.802Z
paper_authors._id: 6613663bcfbba5e761a69531
paper_authors.avatarUrl: /avatars/2baf348371d87a2f9dd4b9c56f1483a9.svg
paper_authors.fullname: Kiwhan Song
paper_authors.type: user
paper_authors.user: kiwhansong
paper_authors._id: 67aac6052c02e43558b6b4b1
paper_authors.name: Boyuan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:34:45.325Z
paper_authors._id: 646935e05d70156639500beb
paper_authors.avatarUrl: /avatars/1373ef2a40145aa505a03bfbae37c95a.svg
paper_authors.fullname: Boyuan Chen
paper_authors.type: user
paper_authors.user: buoyancy99
paper_authors._id: 67aac6052c02e43558b6b4b2
paper_authors.name: Max Simchowitz
paper_authors._id: 67aac6052c02e43558b6b4b3
paper_authors.name: Yilun Du
paper_authors._id: 67aac6052c02e43558b6b4b4
paper_authors.name: Russ Tedrake
paper_authors._id: 67aac6052c02e43558b6b4b5
paper_authors.name: Vincent Sitzmann
paper_title: History-Guided Video Diffusion
paper_summary: Classifier-free guidance (CFG) is a key technique for improving conditional
  generation in diffusion models, enabling more accurate control while enhancing
  sample quality. It is natural to extend this technique to video diffusion,
  which generates video conditioned on a variable number of context frames,
  collectively referred to as history. However, we find two key challenges to
  guiding with variable-length history: architectures that only support
  fixed-size conditioning, and the empirical observation that CFG-style history
  dropout performs poorly. To address this, we propose the Diffusion Forcing
  Transformer (DFoT), a video diffusion architecture and theoretically grounded
  training objective that jointly enable conditioning on a flexible number of
  history frames. We then introduce History Guidance, a family of guidance
  methods uniquely enabled by DFoT. We show that its simplest form, vanilla
  history guidance, already significantly improves video generation quality and
  temporal consistency. A more advanced method, history guidance across time and
  frequency further enhances motion dynamics, enables compositional
  generalization to out-of-distribution history, and can stably roll out
  extremely long videos. Website: https://boyuan.space/history-guidance
paper_ai_keywords: CFG (Classifier-free guidance) diffusion models conditional generation video diffusion context frames history fixed-size conditioning CFG-style history dropout Diffusion Forcing Transformer (DFoT) training objective flexible number of history frames history guidance vanilla history guidance video generation quality temporal consistency history guidance across time and frequency motion dynamics compositional generalization out-of-distribution history rolling out extremely long videos