title: FreeLong: Training-Free Long Video Generation with SpectralBlend
    Temporal Attention
summary: Video diffusion models have made substantial progress in various video
  generation applications. However, training models for long video generation
  tasks require significant computational and data resources, posing a challenge
  to developing long video diffusion models. This paper investigates a
  straightforward and training-free approach to extend an existing short video
  diffusion model (e.g. pre-trained on 16-frame videos) for consistent long video
  generation (e.g. 128 frames). Our preliminary observation has found that
  directly applying the short video diffusion model to generate long videos can
  lead to severe video quality degradation. Further investigation reveals that
  this degradation is primarily due to the distortion of high-frequency
  components in long videos, characterized by a decrease in spatial
  high-frequency components and an increase in temporal high-frequency
  components. Motivated by this, we propose a novel solution named FreeLong to
  balance the frequency distribution of long video features during the denoising
  process. FreeLong blends the low-frequency components of global video features,
  which encapsulate the entire video sequence, with the high-frequency components
  of local video features that focus on shorter subsequences of frames. This
  approach maintains global consistency while incorporating diverse and
  high-quality spatiotemporal details from local videos, enhancing both the
  consistency and fidelity of long video generation. We evaluated FreeLong on
  multiple base video diffusion models and observed significant improvements.
  Additionally, our method supports coherent multi-prompt generation, ensuring
  both visual coherence and seamless transitions between scenes.
paper_authors._id: 66a850b261f6e00f69f6d730
paper_authors.name: Yu Lu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-30T02:32:22.467Z
paper_authors._id: 639ae93c3786549794e97c69
paper_authors.avatarUrl: /avatars/65f7c0b641145f68f22072a5f77e086d.svg
paper_authors.fullname: YL
paper_authors.type: user
paper_authors.user: Simase
paper_authors._id: 66a850b261f6e00f69f6d731
paper_authors.name: Yuanzhi Liang
paper_authors._id: 66a850b261f6e00f69f6d732
paper_authors.name: Linchao Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:58:14.677Z
paper_authors._id: 63521e1dfe367c0d9b155007
paper_authors.avatarUrl: /avatars/b22804fc63b507fd60191486b17cdf7c.svg
paper_authors.fullname: Linchao Zhu
paper_authors.type: user
paper_authors.user: ffmpbgrnn
paper_authors._id: 66a850b261f6e00f69f6d733
paper_authors.name: Yi Yang
paper_title: FreeLong: Training-Free Long Video Generation with SpectralBlend
    Temporal Attention
paper_summary: Video diffusion models have made substantial progress in various video
  generation applications. However, training models for long video generation
  tasks require significant computational and data resources, posing a challenge
  to developing long video diffusion models. This paper investigates a
  straightforward and training-free approach to extend an existing short video
  diffusion model (e.g. pre-trained on 16-frame videos) for consistent long video
  generation (e.g. 128 frames). Our preliminary observation has found that
  directly applying the short video diffusion model to generate long videos can
  lead to severe video quality degradation. Further investigation reveals that
  this degradation is primarily due to the distortion of high-frequency
  components in long videos, characterized by a decrease in spatial
  high-frequency components and an increase in temporal high-frequency
  components. Motivated by this, we propose a novel solution named FreeLong to
  balance the frequency distribution of long video features during the denoising
  process. FreeLong blends the low-frequency components of global video features,
  which encapsulate the entire video sequence, with the high-frequency components
  of local video features that focus on shorter subsequences of frames. This
  approach maintains global consistency while incorporating diverse and
  high-quality spatiotemporal details from local videos, enhancing both the
  consistency and fidelity of long video generation. We evaluated FreeLong on
  multiple base video diffusion models and observed significant improvements.
  Additionally, our method supports coherent multi-prompt generation, ensuring
  both visual coherence and seamless transitions between scenes.
paper_ai_summary: FreeLong enhances long video generation by balancing global and local video features during denoising, improving quality and consistency.
paper_ai_keywords: video diffusion models video generation long video generation short video diffusion model diffusion models temporal high-frequency components spatial high-frequency components denoising global video features local video features spatiotemporal details multi-prompt generation visual coherence seamless transitions