title: Long Video Diffusion Generation with Segmented Cross-Attention and
    Content-Rich Video Data Curation
summary: We introduce Presto, a novel video diffusion model designed to generate
  15-second videos with long-range coherence and rich content. Extending video
  generation methods to maintain scenario diversity over long durations presents
  significant challenges. To address this, we propose a Segmented Cross-Attention
  (SCA) strategy, which splits hidden states into segments along the temporal
  dimension, allowing each segment to cross-attend to a corresponding
  sub-caption. SCA requires no additional parameters, enabling seamless
  incorporation into current DiT-based architectures. To facilitate high-quality
  long video generation, we build the LongTake-HD dataset, consisting of 261k
  content-rich videos with scenario coherence, annotated with an overall video
  caption and five progressive sub-captions. Experiments show that our Presto
  achieves 78.5% on the VBench Semantic Score and 100% on the Dynamic Degree,
  outperforming existing state-of-the-art video generation methods. This
  demonstrates that our proposed Presto significantly enhances content richness,
  maintains long-range coherence, and captures intricate textual details. More
  details are displayed on our project page: https://presto-video.github.io/.
paper_authors._id: 674e81bbc5490201063a1c71
paper_authors.name: Xin Yan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T11:15:08.535Z
paper_authors._id: 62f1fcbee2c9b4f83b4a6276
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f1fcbee2c9b4f83b4a6276/Vb648pHEskN7U0rZouYha.jpeg
paper_authors.fullname: Xin Yan
paper_authors.type: user
paper_authors.user: Cakeyan
paper_authors._id: 674e81bbc5490201063a1c72
paper_authors.name: Yuxuan Cai
paper_authors._id: 674e81bbc5490201063a1c73
paper_authors.name: Qiuyue Wang
paper_authors._id: 674e81bbc5490201063a1c74
paper_authors.name: Yuan Zhou
paper_authors._id: 674e81bbc5490201063a1c75
paper_authors.name: Wenhao Huang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-03T03:57:51.296Z
paper_authors._id: 639005861c5a6237272557d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1670382915881-noauth.jpeg
paper_authors.fullname: Huan Yang
paper_authors.type: user
paper_authors.user: hyang0511
paper_authors._id: 674e81bbc5490201063a1c76
paper_authors.name: Huan Yang
paper_title: Long Video Diffusion Generation with Segmented Cross-Attention and
    Content-Rich Video Data Curation
paper_summary: We introduce Presto, a novel video diffusion model designed to generate
  15-second videos with long-range coherence and rich content. Extending video
  generation methods to maintain scenario diversity over long durations presents
  significant challenges. To address this, we propose a Segmented Cross-Attention
  (SCA) strategy, which splits hidden states into segments along the temporal
  dimension, allowing each segment to cross-attend to a corresponding
  sub-caption. SCA requires no additional parameters, enabling seamless
  incorporation into current DiT-based architectures. To facilitate high-quality
  long video generation, we build the LongTake-HD dataset, consisting of 261k
  content-rich videos with scenario coherence, annotated with an overall video
  caption and five progressive sub-captions. Experiments show that our Presto
  achieves 78.5% on the VBench Semantic Score and 100% on the Dynamic Degree,
  outperforming existing state-of-the-art video generation methods. This
  demonstrates that our proposed Presto significantly enhances content richness,
  maintains long-range coherence, and captures intricate textual details. More
  details are displayed on our project page: https://presto-video.github.io/.
paper_ai_summary: Presto, a novel video diffusion model with a segmented cross-attention strategy, generates high-quality 15-second videos with content richness and long-range coherence, outperforming existing methods.
paper_ai_keywords: video diffusion model Segmented Cross-Attention (SCA) DiT-based architectures LongTake-HD dataset VBench Semantic Score Dynamic Degree