title: SageAttention2++: A More Efficient Implementation of SageAttention2
summary: The efficiency of attention is critical because its time complexity grows
  quadratically with sequence length. SageAttention2 addresses this by utilizing
  quantization to accelerate matrix multiplications (Matmul) in attention. To
  further accelerate SageAttention2, we propose to utilize the faster instruction
  of FP8 Matmul accumulated in FP16. The instruction is 2x faster than the FP8
  Matmul used in SageAttention2. Our experiments show that SageAttention2++
  achieves a 3.9x speedup over FlashAttention while maintaining the same
  attention accuracy as SageAttention2. This means SageAttention2++ effectively
  accelerates various models, including those for language, image, and video
  generation, with negligible end-to-end metrics loss. The code will be available
  at https://github.com/thu-ml/SageAttention.
paper_authors._id: 6837c91ec790885f338b8f27
paper_authors.name: Jintao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:42:22.525Z
paper_authors._id: 66c0a08bac74db25de8427ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
paper_authors.fullname: Jintao Zhang
paper_authors.type: user
paper_authors.user: jt-zhang
paper_authors._id: 6837c91ec790885f338b8f28
paper_authors.name: Xiaoming Xu
paper_authors._id: 6837c91ec790885f338b8f29
paper_authors.name: Jia Wei
paper_authors._id: 6837c91ec790885f338b8f2a
paper_authors.name: Haofeng Huang
paper_authors._id: 6837c91ec790885f338b8f2b
paper_authors.name: Pengle Zhang
paper_authors._id: 6837c91ec790885f338b8f2c
paper_authors.name: Chendong Xiang
paper_authors._id: 6837c91ec790885f338b8f2d
paper_authors.name: Jun Zhu
paper_authors._id: 6837c91ec790885f338b8f2e
paper_authors.name: Jianfei Chen
paper_title: SageAttention2++: A More Efficient Implementation of SageAttention2
paper_summary: The efficiency of attention is critical because its time complexity grows
  quadratically with sequence length. SageAttention2 addresses this by utilizing
  quantization to accelerate matrix multiplications (Matmul) in attention. To
  further accelerate SageAttention2, we propose to utilize the faster instruction
  of FP8 Matmul accumulated in FP16. The instruction is 2x faster than the FP8
  Matmul used in SageAttention2. Our experiments show that SageAttention2++
  achieves a 3.9x speedup over FlashAttention while maintaining the same
  attention accuracy as SageAttention2. This means SageAttention2++ effectively
  accelerates various models, including those for language, image, and video
  generation, with negligible end-to-end metrics loss. The code will be available
  at https://github.com/thu-ml/SageAttention.
paper_ai_summary: SageAttention2++ improves attention efficiency by using FP8 Matmul in FP16, achieving a 3.9x speedup over FlashAttention without losing accuracy.
paper_ai_keywords: attention time complexity sequence length quantization matrix multiplications Matmul FP8 FP16 SageAttention2 SageAttention2++ FlashAttention image generation video generation