title: BroadWay: Boost Your Text-to-Video Generation Model in a Training-free
    Way
summary: The text-to-video (T2V) generation models, offering convenient visual
  creation, have recently garnered increasing attention. Despite their
  substantial potential, the generated videos may present artifacts, including
  structural implausibility, temporal inconsistency, and a lack of motion, often
  resulting in near-static video. In this work, we have identified a correlation
  between the disparity of temporal attention maps across different blocks and
  the occurrence of temporal inconsistencies. Additionally, we have observed that
  the energy contained within the temporal attention maps is directly related to
  the magnitude of motion amplitude in the generated videos. Based on these
  observations, we present BroadWay, a training-free method to improve the
  quality of text-to-video generation without introducing additional parameters,
  augmenting memory or sampling time. Specifically, BroadWay is composed of two
  principal components: 1) Temporal Self-Guidance improves the structural
  plausibility and temporal consistency of generated videos by reducing the
  disparity between the temporal attention maps across various decoder blocks. 2)
  Fourier-based Motion Enhancement enhances the magnitude and richness of motion
  by amplifying the energy of the map. Extensive experiments demonstrate that
  BroadWay significantly improves the quality of text-to-video generation with
  negligible additional cost.
paper_authors._id: 670749315ea108eecd14ec6f
paper_authors.name: Jiazi Bu
paper_authors._id: 670749315ea108eecd14ec70
paper_authors.name: Pengyang Ling
paper_authors._id: 670749315ea108eecd14ec71
paper_authors.name: Pan Zhang
paper_authors._id: 670749315ea108eecd14ec72
paper_authors.name: Tong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:21:50.443Z
paper_authors._id: 6310929fda0aaad2f70397a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: wutong16
paper_authors._id: 670749315ea108eecd14ec73
paper_authors.name: Xiaoyi Dong
paper_authors._id: 670749315ea108eecd14ec74
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:56:05.632Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/iUQm5FAomzqYi6fkqIn9F.jpeg
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 670749315ea108eecd14ec75
paper_authors.name: Yuhang Cao
paper_authors._id: 670749315ea108eecd14ec76
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:56:35.238Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 670749315ea108eecd14ec77
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:21:48.196Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: BroadWay: Boost Your Text-to-Video Generation Model in a Training-free
    Way
paper_summary: The text-to-video (T2V) generation models, offering convenient visual
  creation, have recently garnered increasing attention. Despite their
  substantial potential, the generated videos may present artifacts, including
  structural implausibility, temporal inconsistency, and a lack of motion, often
  resulting in near-static video. In this work, we have identified a correlation
  between the disparity of temporal attention maps across different blocks and
  the occurrence of temporal inconsistencies. Additionally, we have observed that
  the energy contained within the temporal attention maps is directly related to
  the magnitude of motion amplitude in the generated videos. Based on these
  observations, we present BroadWay, a training-free method to improve the
  quality of text-to-video generation without introducing additional parameters,
  augmenting memory or sampling time. Specifically, BroadWay is composed of two
  principal components: 1) Temporal Self-Guidance improves the structural
  plausibility and temporal consistency of generated videos by reducing the
  disparity between the temporal attention maps across various decoder blocks. 2)
  Fourier-based Motion Enhancement enhances the magnitude and richness of motion
  by amplifying the energy of the map. Extensive experiments demonstrate that
  BroadWay significantly improves the quality of text-to-video generation with
  negligible additional cost.
paper_ai_keywords: text-to-video (T2V) generation models artifacts reconstruction implplausibility temporal inconsistency motion temporal attention maps decoder blocks Temporal Self-Guidance Fourier-based Motion Enhancement magnitude motion amplitude generated videos optimization plausibility temporal consistency disparity