title: RepVideo: Rethinking Cross-Layer Representation for Video Generation
summary: Video generation has achieved remarkable progress with the introduction of
  diffusion models, which have significantly improved the quality of generated
  videos. However, recent research has primarily focused on scaling up model
  training, while offering limited insights into the direct impact of
  representations on the video generation process. In this paper, we initially
  investigate the characteristics of features in intermediate layers, finding
  substantial variations in attention maps across different layers. These
  variations lead to unstable semantic representations and contribute to
  cumulative differences between features, which ultimately reduce the similarity
  between adjacent frames and negatively affect temporal coherence. To address
  this, we propose RepVideo, an enhanced representation framework for
  text-to-video diffusion models. By accumulating features from neighboring
  layers to form enriched representations, this approach captures more stable
  semantic information. These enhanced representations are then used as inputs to
  the attention mechanism, thereby improving semantic expressiveness while
  ensuring feature consistency across adjacent frames. Extensive experiments
  demonstrate that our RepVideo not only significantly enhances the ability to
  generate accurate spatial appearances, such as capturing complex spatial
  relationships between multiple objects, but also improves temporal consistency
  in video generation.
paper_authors._id: 6788945e2b5050a9154d939d
paper_authors.name: Chenyang Si
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:49:14.615Z
paper_authors._id: 635f8ed47c05eb9f59963d3a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635f8ed47c05eb9f59963d3a/uQf4p9N9pSaFy87Wg9v4k.jpeg
paper_authors.fullname: ChenyangSi
paper_authors.type: user
paper_authors.user: ChenyangSi
paper_authors._id: 6788945e2b5050a9154d939e
paper_authors.name: Weichen Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:49:38.866Z
paper_authors._id: 6481764e8af4675862efb22e
paper_authors.avatarUrl: /avatars/fc2e076bc861693f598a528a068a696e.svg
paper_authors.fullname: weichenfan
paper_authors.type: user
paper_authors.user: weepiess2383
paper_authors._id: 6788945e2b5050a9154d939f
paper_authors.name: Zhengyao Lv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:49:45.309Z
paper_authors._id: 645aff5121ab438e732c47c1
paper_authors.avatarUrl: /avatars/23b2a853139b0f2ae1fa88e2bd4e0056.svg
paper_authors.fullname: Zhengyao Lv
paper_authors.type: user
paper_authors.user: cszy98
paper_authors._id: 6788945e2b5050a9154d93a0
paper_authors.name: Ziqi Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:49:51.123Z
paper_authors._id: 60efe7fa0d920bc7805cada5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60efe7fa0d920bc7805cada5/2LBrJBjSCOP5ilZIpWLHl.png
paper_authors.fullname: Ziqi Huang
paper_authors.type: user
paper_authors.user: Ziqi
paper_authors._id: 6788945e2b5050a9154d93a1
paper_authors.name: Yu Qiao
paper_authors._id: 6788945e2b5050a9154d93a2
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:49:58.339Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_title: RepVideo: Rethinking Cross-Layer Representation for Video Generation
paper_summary: Video generation has achieved remarkable progress with the introduction of
  diffusion models, which have significantly improved the quality of generated
  videos. However, recent research has primarily focused on scaling up model
  training, while offering limited insights into the direct impact of
  representations on the video generation process. In this paper, we initially
  investigate the characteristics of features in intermediate layers, finding
  substantial variations in attention maps across different layers. These
  variations lead to unstable semantic representations and contribute to
  cumulative differences between features, which ultimately reduce the similarity
  between adjacent frames and negatively affect temporal coherence. To address
  this, we propose RepVideo, an enhanced representation framework for
  text-to-video diffusion models. By accumulating features from neighboring
  layers to form enriched representations, this approach captures more stable
  semantic information. These enhanced representations are then used as inputs to
  the attention mechanism, thereby improving semantic expressiveness while
  ensuring feature consistency across adjacent frames. Extensive experiments
  demonstrate that our RepVideo not only significantly enhances the ability to
  generate accurate spatial appearances, such as capturing complex spatial
  relationships between multiple objects, but also improves temporal consistency
  in video generation.
paper_ai_summary: RepVideo enhances video generation by stabilizing semantic representations through enriched feature accumulation, improving spatial accuracy and temporal coherence in text-to-video diffusion models.
paper_ai_keywords: diffusion models attention maps semantic representations temporal coherence RepVideo feature accumulation spatial appearances temporal consistency