title: Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via
    Semantic-Aware Permutation
summary: Diffusion Transformers (DiTs) are essential for video generation but suffer
  from significant latency due to the quadratic complexity of attention. By
  computing only critical tokens, sparse attention reduces computational costs
  and offers a promising acceleration approach. However, we identify that
  existing methods fail to approach optimal generation quality under the same
  computation budget for two reasons: (1) Inaccurate critical token
  identification: current methods cluster tokens based on position rather than
  semantics, leading to imprecise aggregated representations. (2) Excessive
  computation waste: critical tokens are scattered among non-critical ones,
  leading to wasted computation on GPUs, which are optimized for processing
  contiguous tokens. In this paper, we propose SVG2, a training-free framework
  that maximizes identification accuracy and minimizes computation waste,
  achieving a Pareto frontier trade-off between generation quality and
  efficiency. The core of SVG2 is semantic-aware permutation, which clusters and
  reorders tokens based on semantic similarity using k-means. This approach
  ensures both a precise cluster representation, improving identification
  accuracy, and a densified layout of critical tokens, enabling efficient
  computation without padding. Additionally, SVG2 integrates top-p dynamic budget
  control and customized kernel implementations, achieving up to 2.30x and 1.89x
  speedup while maintaining a PSNR of up to 30 and 26 on HunyuanVideo and Wan
  2.1, respectively.
paper_authors._id: 683536db6d3dc82656b13765
paper_authors.name: Shuo Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:50:23.533Z
paper_authors._id: 642b970ceb31218a5f204a29
paper_authors.avatarUrl: /avatars/582287f477bbb1a0842787145e375fd3.svg
paper_authors.fullname: andy-yang
paper_authors.type: user
paper_authors.user: andy-yang
paper_authors._id: 683536db6d3dc82656b13766
paper_authors.name: Haocheng Xi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:49:30.035Z
paper_authors._id: 66ce751a8ec9fda2cf5a9e85
paper_authors.avatarUrl: /avatars/c17093ca81dad007b3e50bae503955a7.svg
paper_authors.fullname: Haocheng Xi
paper_authors.type: user
paper_authors.user: xihc-ucb
paper_authors._id: 683536db6d3dc82656b13767
paper_authors.name: Yilong Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:49:32.515Z
paper_authors._id: 6549b0a808775ce78e535c6a
paper_authors.avatarUrl: /avatars/942066356843d0c424375937f157c975.svg
paper_authors.fullname: Yilong Zhao
paper_authors.type: user
paper_authors.user: ylzhao
paper_authors._id: 683536db6d3dc82656b13768
paper_authors.name: Muyang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:44:05.243Z
paper_authors._id: 63129589bbaa385279d1826e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63129589bbaa385279d1826e/0AUs3q4ngRZ-wXuY1jP9G.jpeg
paper_authors.fullname: Muyang Li
paper_authors.type: user
paper_authors.user: Lmxyy
paper_authors._id: 683536db6d3dc82656b13769
paper_authors.name: Jintao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:28.809Z
paper_authors._id: 66c0a08bac74db25de8427ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
paper_authors.fullname: Jintao Zhang
paper_authors.type: user
paper_authors.user: jt-zhang
paper_authors._id: 683536db6d3dc82656b1376a
paper_authors.name: Han Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:30.674Z
paper_authors._id: 650e2b14c945dfc9386a7e28
paper_authors.avatarUrl: /avatars/0a9be20aa53f4c52a2d1a8b02d4093ea.svg
paper_authors.fullname: Han Cai
paper_authors.type: user
paper_authors.user: han-cai
paper_authors._id: 683536db6d3dc82656b1376b
paper_authors.name: Yujun Lin
paper_authors._id: 683536db6d3dc82656b1376c
paper_authors.name: Xiuyu Li
paper_authors._id: 683536db6d3dc82656b1376d
paper_authors.name: Chenfeng Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:04:22.005Z
paper_authors._id: 654ca71d5255ee86711b52c5
paper_authors.avatarUrl: /avatars/52bf00fd74c8db5643c4daa185c678e6.svg
paper_authors.fullname: Chenfeng Xu
paper_authors.type: user
paper_authors.user: chenfengx
paper_authors._id: 683536db6d3dc82656b1376e
paper_authors.name: Kelly Peng
paper_authors._id: 683536db6d3dc82656b1376f
paper_authors.name: Jianfei Chen
paper_authors._id: 683536db6d3dc82656b13770
paper_authors.name: Song Han
paper_authors._id: 683536db6d3dc82656b13771
paper_authors.name: Kurt Keutzer
paper_authors._id: 683536db6d3dc82656b13772
paper_authors.name: Ion Stoica
paper_title: Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via
    Semantic-Aware Permutation
paper_summary: Diffusion Transformers (DiTs) are essential for video generation but suffer
  from significant latency due to the quadratic complexity of attention. By
  computing only critical tokens, sparse attention reduces computational costs
  and offers a promising acceleration approach. However, we identify that
  existing methods fail to approach optimal generation quality under the same
  computation budget for two reasons: (1) Inaccurate critical token
  identification: current methods cluster tokens based on position rather than
  semantics, leading to imprecise aggregated representations. (2) Excessive
  computation waste: critical tokens are scattered among non-critical ones,
  leading to wasted computation on GPUs, which are optimized for processing
  contiguous tokens. In this paper, we propose SVG2, a training-free framework
  that maximizes identification accuracy and minimizes computation waste,
  achieving a Pareto frontier trade-off between generation quality and
  efficiency. The core of SVG2 is semantic-aware permutation, which clusters and
  reorders tokens based on semantic similarity using k-means. This approach
  ensures both a precise cluster representation, improving identification
  accuracy, and a densified layout of critical tokens, enabling efficient
  computation without padding. Additionally, SVG2 integrates top-p dynamic budget
  control and customized kernel implementations, achieving up to 2.30x and 1.89x
  speedup while maintaining a PSNR of up to 30 and 26 on HunyuanVideo and Wan
  2.1, respectively.
paper_ai_summary: SVG2 is a training-free framework that enhances video generation efficiency and quality by accurately identifying and processing critical tokens using semantic-aware permutation and dynamic budget control.
paper_ai_keywords: Diffusion Transformers sparse attention critical tokens semantic similarity semantic-aware permutation k-means top-p dynamic budget control