EM Distillation for One-step Diffusion Models
While diffusion models can learn complex distributions, sampling requires a
computationally expensive iterative process. Existing distillation methods
enable efficient sampling, but have notable limitations, such as performance
degradation with very few sampling steps, reliance on training data access, or
mode-seeking optimization that may fail to capture the full distribution. We
propose EM Distillation (EMD), a maximum likelihood-based approach that
distills a diffusion model to a one-step generator model with minimal loss of
perceptual quality. Our approach is derived through the lens of
Expectation-Maximization (EM), where the generator parameters are updated using
samples from the joint distribution of the diffusion teacher prior and inferred
generator latents. We develop a reparametrized sampling scheme and a noise
cancellation technique that together stabilizes the distillation process. We
further reveal an interesting connection of our method with existing methods
that minimize mode-seeking KL. EMD outperforms existing one-step generative
methods in terms of FID scores on ImageNet-64 and ImageNet-128, and compares
favorably with prior work on distilling text-to-image diffusion models.
66554e51fda461267ce7ad82
Sirui Xie
admin_assigned
2024-05-28T10:32:10.673Z
654b3971b3f4b6f747e590a0
/avatars/103fd588070a306cf624e27a678cd0d5.svg
Sirui Xie
user
SiriusXie
66554e51fda461267ce7ad83
Zhisheng Xiao
admin_assigned
2024-05-28T10:32:19.947Z
630846e82d33960e6b7a43ce
/avatars/058ad9a17d51ce9e1952d2677b0a47bc.svg
Xiao
user
Zhisheng
66554e51fda461267ce7ad84
Diederik P Kingma
admin_assigned
2024-05-28T10:32:25.690Z
630874fd6fb2ea4413f0dd78
/avatars/aac7fe259f341655b34a3381ee197226.svg
Diederik P Kingma
user
dpkingma
66554e51fda461267ce7ad85
Tingbo Hou
admin_assigned
2024-05-28T10:32:34.034Z
655846d7ed8df83128f5826a
/avatars/d7ce174d7d1b8614d5f6f071225c0057.svg
Hou
user
Tingbo
66554e51fda461267ce7ad86
Ying Nian Wu
66554e51fda461267ce7ad87
Kevin Patrick Murphy
66554e51fda461267ce7ad88
Tim Salimans
66554e51fda461267ce7ad89
Ben Poole
admin_assigned
2024-05-28T10:33:19.776Z
63213768d2d45f315183abc3
/avatars/521f52f430b73e514afb06e3a9d6779d.svg
Ben Poole
user
doinkda
66554e51fda461267ce7ad8a
Ruiqi Gao
admin_assigned
2024-05-28T10:33:30.403Z
6137d46d1c0a1ca53676925f
/avatars/f3d884a3971a858adc49e0f3b1932078.svg
Gao
user
Ruiqi
EM Distillation for One-step Diffusion Models
While diffusion models can learn complex distributions, sampling requires a
computationally expensive iterative process. Existing distillation methods
enable efficient sampling, but have notable limitations, such as performance
degradation with very few sampling steps, reliance on training data access, or
mode-seeking optimization that may fail to capture the full distribution. We
propose EM Distillation (EMD), a maximum likelihood-based approach that
distills a diffusion model to a one-step generator model with minimal loss of
perceptual quality. Our approach is derived through the lens of
Expectation-Maximization (EM), where the generator parameters are updated using
samples from the joint distribution of the diffusion teacher prior and inferred
generator latents. We develop a reparametrized sampling scheme and a noise
cancellation technique that together stabilizes the distillation process. We
further reveal an interesting connection of our method with existing methods
that minimize mode-seeking KL. EMD outperforms existing one-step generative
methods in terms of FID scores on ImageNet-64 and ImageNet-128, and compares
favorably with prior work on distilling text-to-image diffusion models.
EM Distillation stabilizes the distillation of diffusion models into one-step generators, enhancing perceptual quality and outperforming existing methods in FID scores and text-to-image tasks.
diffusion models sampling distillation Expectation-Maximization reparametrized sampling noise cancellation FID scores ImageNet-64 ImageNet-128 text-to-image diffusion models