Scale-wise Distillation of Diffusion Models
We present SwD, a scale-wise distillation framework for diffusion models
(DMs), which effectively employs next-scale prediction ideas for
diffusion-based few-step generators. In more detail, SwD is inspired by the
recent insights relating diffusion processes to the implicit spectral
autoregression. We suppose that DMs can initiate generation at lower data
resolutions and gradually upscale the samples at each denoising step without
loss in performance while significantly reducing computational costs. SwD
naturally integrates this idea into existing diffusion distillation methods
based on distribution matching. Also, we enrich the family of distribution
matching approaches by introducing a novel patch loss enforcing finer-grained
similarity to the target distribution. When applied to state-of-the-art
text-to-image diffusion models, SwD approaches the inference times of two full
resolution steps and significantly outperforms the counterparts under the same
computation budget, as evidenced by automated metrics and human preference
studies.
67dd1227046f2c38458e9588
Nikita Starodubcev
claimed_verified
2025-03-24T08:53:21.020Z
6410d3a4cfbe9c4400233d1e
/avatars/9e1c4e0ea5f6964c90013f6e19c41db1.svg
nikita
user
quickjkee
67dd1227046f2c38458e9589
Denis Kuznedelev
admin_assigned
2025-03-21T12:59:13.018Z
629cf0475a13ba8233dd18c9
https://cdn-avatars.huggingface.co/v1/production/uploads/1654452258405-noauth.jpeg
Denis Kuznedelev
user
SpiridonSunRotator
67dd1227046f2c38458e958a
Artem Babenko
67dd1227046f2c38458e958b
Dmitry Baranchuk
admin_assigned
2025-03-21T12:59:23.033Z
62b6cc49752323892323bc04
https://cdn-avatars.huggingface.co/v1/production/uploads/62b6cc49752323892323bc04/gGBld1KJIP9AIpd81L3PC.jpeg
Dmitry Baranchuk
user
dbaranchuk
Scale-wise Distillation of Diffusion Models
We present SwD, a scale-wise distillation framework for diffusion models
(DMs), which effectively employs next-scale prediction ideas for
diffusion-based few-step generators. In more detail, SwD is inspired by the
recent insights relating diffusion processes to the implicit spectral
autoregression. We suppose that DMs can initiate generation at lower data
resolutions and gradually upscale the samples at each denoising step without
loss in performance while significantly reducing computational costs. SwD
naturally integrates this idea into existing diffusion distillation methods
based on distribution matching. Also, we enrich the family of distribution
matching approaches by introducing a novel patch loss enforcing finer-grained
similarity to the target distribution. When applied to state-of-the-art
text-to-image diffusion models, SwD approaches the inference times of two full
resolution steps and significantly outperforms the counterparts under the same
computation budget, as evidenced by automated metrics and human preference
studies.
A scale-wise distillation framework for diffusion models reduces computational costs and improves inference times by incorporating next-scale predictions and enhancing distribution matching methods.
diffusion models scale-wise distillation next-scale prediction implicit spectral autoregression denoising computational costs distribution matching patch loss inference times text-to-image diffusion models