SageAttention2++: A More Efficient Implementation of SageAttention2
The efficiency of attention is critical because its time complexity grows
quadratically with sequence length. SageAttention2 addresses this by utilizing
quantization to accelerate matrix multiplications (Matmul) in attention. To
further accelerate SageAttention2, we propose to utilize the faster instruction
of FP8 Matmul accumulated in FP16. The instruction is 2x faster than the FP8
Matmul used in SageAttention2. Our experiments show that SageAttention2++
achieves a 3.9x speedup over FlashAttention while maintaining the same
attention accuracy as SageAttention2. This means SageAttention2++ effectively
accelerates various models, including those for language, image, and video
generation, with negligible end-to-end metrics loss. The code will be available
at https://github.com/thu-ml/SageAttention.
6837c91ec790885f338b8f27
Jintao Zhang
claimed_verified
2025-05-29T07:42:22.525Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
user
jt-zhang
6837c91ec790885f338b8f28
Xiaoming Xu
6837c91ec790885f338b8f29
Jia Wei
6837c91ec790885f338b8f2a
Haofeng Huang
6837c91ec790885f338b8f2b
Pengle Zhang
6837c91ec790885f338b8f2c
Chendong Xiang
6837c91ec790885f338b8f2d
Jun Zhu
6837c91ec790885f338b8f2e
Jianfei Chen
SageAttention2++: A More Efficient Implementation of SageAttention2
The efficiency of attention is critical because its time complexity grows
quadratically with sequence length. SageAttention2 addresses this by utilizing
quantization to accelerate matrix multiplications (Matmul) in attention. To
further accelerate SageAttention2, we propose to utilize the faster instruction
of FP8 Matmul accumulated in FP16. The instruction is 2x faster than the FP8
Matmul used in SageAttention2. Our experiments show that SageAttention2++
achieves a 3.9x speedup over FlashAttention while maintaining the same
attention accuracy as SageAttention2. This means SageAttention2++ effectively
accelerates various models, including those for language, image, and video
generation, with negligible end-to-end metrics loss. The code will be available
at https://github.com/thu-ml/SageAttention.
SageAttention2++ improves attention efficiency by using FP8 Matmul in FP16, achieving a 3.9x speedup over FlashAttention without losing accuracy.
attention time complexity sequence length quantization matrix multiplications Matmul FP8 FP16 SageAttention2 SageAttention2++ FlashAttention image generation video generation