QVGen: Pushing the Limit of Quantized Video Generative Models
Video diffusion models (DMs) have enabled high-quality video synthesis. Yet,
their substantial computational and memory demands pose serious challenges to
real-world deployment, even on high-end GPUs. As a commonly adopted solution,
quantization has proven notable success in reducing cost for image DMs, while
its direct application to video DMs remains ineffective. In this paper, we
present QVGen, a novel quantization-aware training (QAT) framework tailored for
high-performance and inference-efficient video DMs under extremely low-bit
quantization (e.g., 4-bit or below). We begin with a theoretical analysis
demonstrating that reducing the gradient norm is essential to facilitate
convergence for QAT. To this end, we introduce auxiliary modules (Phi) to
mitigate large quantization errors, leading to significantly enhanced
convergence. To eliminate the inference overhead of Phi, we propose a
rank-decay strategy that progressively eliminates Phi. Specifically, we
repeatedly employ singular value decomposition (SVD) and a proposed rank-based
regularization gamma to identify and decay low-contributing
components. This strategy retains performance while zeroing out inference
overhead. Extensive experiments across 4 state-of-the-art (SOTA) video DMs,
with parameter sizes ranging from 1.3B sim14B, show that QVGen is the
first to reach full-precision comparable quality under 4-bit settings.
Moreover, it significantly outperforms existing methods. For instance, our
3-bit CogVideoX-2B achieves improvements of +25.28 in Dynamic Degree and
+8.43 in Scene Consistency on VBench.
682c27b10f622b7afc25df1f
Yushi Huang
claimed_verified
2025-05-20T08:07:25.265Z
64b500fdf460afaefc5c64b3
/avatars/0cb90e3fdd116e1a49209b222125c76e.svg
Yushi Huang
user
Harahan
682c27b10f622b7afc25df20
Ruihao Gong
admin_assigned
2025-05-20T08:53:49.624Z
648876a7063b5020501479f0
/avatars/0a8a0c1d4ebf8e444d151e634d55e91f.svg
Gong
user
Ruihao
682c27b10f622b7afc25df21
Jing Liu
682c27b10f622b7afc25df22
Yifu Ding
682c27b10f622b7afc25df23
Chengtao Lv
admin_assigned
2025-05-20T08:53:14.848Z
64e9bfc3f494f8b2a061a010
/avatars/e55cfea55b45b03d1abfa38db6af58b6.svg
吕呈滔
user
lvchengtao
682c27b10f622b7afc25df24
Haotong Qin
admin_assigned
2025-05-20T08:52:56.938Z
65c49589c0b1921e19260a8d
/avatars/7ce9af8c627f2a0c3db6bde82290ee1f.svg
Haotong Qin
user
HaotongQin
682c27b10f622b7afc25df25
Jun Zhang
QVGen: Pushing the Limit of Quantized Video Generative Models
Video diffusion models (DMs) have enabled high-quality video synthesis. Yet,
their substantial computational and memory demands pose serious challenges to
real-world deployment, even on high-end GPUs. As a commonly adopted solution,
quantization has proven notable success in reducing cost for image DMs, while
its direct application to video DMs remains ineffective. In this paper, we
present QVGen, a novel quantization-aware training (QAT) framework tailored for
high-performance and inference-efficient video DMs under extremely low-bit
quantization (e.g., 4-bit or below). We begin with a theoretical analysis
demonstrating that reducing the gradient norm is essential to facilitate
convergence for QAT. To this end, we introduce auxiliary modules (Phi) to
mitigate large quantization errors, leading to significantly enhanced
convergence. To eliminate the inference overhead of Phi, we propose a
rank-decay strategy that progressively eliminates Phi. Specifically, we
repeatedly employ singular value decomposition (SVD) and a proposed rank-based
regularization gamma to identify and decay low-contributing
components. This strategy retains performance while zeroing out inference
overhead. Extensive experiments across 4 state-of-the-art (SOTA) video DMs,
with parameter sizes ranging from 1.3B sim14B, show that QVGen is the
first to reach full-precision comparable quality under 4-bit settings.
Moreover, it significantly outperforms existing methods. For instance, our
3-bit CogVideoX-2B achieves improvements of +25.28 in Dynamic Degree and
+8.43 in Scene Consistency on VBench.
QVGen is a quantization-aware training framework that enhances the performance and efficiency of video diffusion models under low-bit quantization, achieving high-quality video synthesis comparable to full-precision models.
video diffusion models quantization quantization-aware training QAT gradient norm auxiliary modules singular value decomposition rank-based regularization Dynamic Degree Scene Consistency