Learning Few-Step Diffusion Models by Trajectory Distribution Matching
Accelerating diffusion model sampling is crucial for efficient AIGC
deployment. While diffusion distillation methods -- based on distribution
matching and trajectory matching -- reduce sampling to as few as one step, they
fall short on complex tasks like text-to-image generation. Few-step generation
offers a better balance between speed and quality, but existing approaches face
a persistent trade-off: distribution matching lacks flexibility for multi-step
sampling, while trajectory matching often yields suboptimal image quality. To
bridge this gap, we propose learning few-step diffusion models by Trajectory
Distribution Matching (TDM), a unified distillation paradigm that combines the
strengths of distribution and trajectory matching. Our method introduces a
data-free score distillation objective, aligning the student's trajectory with
the teacher's at the distribution level. Further, we develop a
sampling-steps-aware objective that decouples learning targets across different
steps, enabling more adjustable sampling. This approach supports both
deterministic sampling for superior image quality and flexible multi-step
adaptation, achieving state-of-the-art performance with remarkable efficiency.
Our model, TDM, outperforms existing methods on various backbones, such as SDXL
and PixArt-alpha, delivering superior quality and significantly reduced
training costs. In particular, our method distills PixArt-alpha into a
4-step generator that outperforms its teacher on real user preference at 1024
resolution. This is accomplished with 500 iterations and 2 A800 hours -- a mere
0.01% of the teacher's training cost. In addition, our proposed TDM can be
extended to accelerate text-to-video diffusion. Notably, TDM can outperform its
teacher model (CogVideoX-2B) by using only 4 NFE on VBench, improving the total
score from 80.91 to 81.65. Project page: https://tdm-t2x.github.io/
67d6881cf997964e21f90598
Yihong Luo
claimed_verified
2025-03-17T08:44:52.452Z
65f7e6856bd4bac5b6a4ecc3
/avatars/9c0c48310fb5e99c19700316a0ab531e.svg
Yihong Luo
user
Luo-Yihong
67d6881cf997964e21f90599
Tianyang Hu
admin_assigned
2025-03-17T08:56:48.283Z
636a40faa6f948c4f0c62ae5
/avatars/30c35b194ba84d6e274df30e91a8cc45.svg
Tianyang Hu
user
whatlegequ
67d6881cf997964e21f9059a
Jiacheng Sun
admin_assigned
2025-03-17T08:56:55.114Z
67b91a3c186bc4f8d83c94cf
/avatars/a79538be4b5ed02cd54556458375e4af.svg
Jiacheng Sun
user
JIACSUN96
67d6881cf997964e21f9059b
Yujun Cai
67d6881cf997964e21f9059c
Jing Tang
admin_assigned
2025-03-17T08:57:09.362Z
636d660056c0762cfd9dc8d5
/avatars/50ea2100e00b67ef10adc57556477184.svg
jing tang
user
jingtang
Learning Few-Step Diffusion Models by Trajectory Distribution Matching
Accelerating diffusion model sampling is crucial for efficient AIGC
deployment. While diffusion distillation methods -- based on distribution
matching and trajectory matching -- reduce sampling to as few as one step, they
fall short on complex tasks like text-to-image generation. Few-step generation
offers a better balance between speed and quality, but existing approaches face
a persistent trade-off: distribution matching lacks flexibility for multi-step
sampling, while trajectory matching often yields suboptimal image quality. To
bridge this gap, we propose learning few-step diffusion models by Trajectory
Distribution Matching (TDM), a unified distillation paradigm that combines the
strengths of distribution and trajectory matching. Our method introduces a
data-free score distillation objective, aligning the student's trajectory with
the teacher's at the distribution level. Further, we develop a
sampling-steps-aware objective that decouples learning targets across different
steps, enabling more adjustable sampling. This approach supports both
deterministic sampling for superior image quality and flexible multi-step
adaptation, achieving state-of-the-art performance with remarkable efficiency.
Our model, TDM, outperforms existing methods on various backbones, such as SDXL
and PixArt-alpha, delivering superior quality and significantly reduced
training costs. In particular, our method distills PixArt-alpha into a
4-step generator that outperforms its teacher on real user preference at 1024
resolution. This is accomplished with 500 iterations and 2 A800 hours -- a mere
0.01% of the teacher's training cost. In addition, our proposed TDM can be
extended to accelerate text-to-video diffusion. Notably, TDM can outperform its
teacher model (CogVideoX-2B) by using only 4 NFE on VBench, improving the total
score from 80.91 to 81.65. Project page: https://tdm-t2x.github.io/
TDM, a Trajectory Distribution Matching method, accelerates diffusion model sampling by combining distribution and trajectory matching, achieving superior quality and efficiency in text-to-image and text-to-video generation.
diffusion model sampling diffusion distillation distribution matching trajectory matching few-step generation data-free score distillation sampling-steps-aware objective deterministic sampling SDXL PixArt-$\alpha$ text-to-video diffusion CogVideoX-2B VBench TDM