title: EM Distillation for One-step Diffusion Models
summary: While diffusion models can learn complex distributions, sampling requires a
  computationally expensive iterative process. Existing distillation methods
  enable efficient sampling, but have notable limitations, such as performance
  degradation with very few sampling steps, reliance on training data access, or
  mode-seeking optimization that may fail to capture the full distribution. We
  propose EM Distillation (EMD), a maximum likelihood-based approach that
  distills a diffusion model to a one-step generator model with minimal loss of
  perceptual quality. Our approach is derived through the lens of
  Expectation-Maximization (EM), where the generator parameters are updated using
  samples from the joint distribution of the diffusion teacher prior and inferred
  generator latents. We develop a reparametrized sampling scheme and a noise
  cancellation technique that together stabilizes the distillation process. We
  further reveal an interesting connection of our method with existing methods
  that minimize mode-seeking KL. EMD outperforms existing one-step generative
  methods in terms of FID scores on ImageNet-64 and ImageNet-128, and compares
  favorably with prior work on distilling text-to-image diffusion models.
paper_authors._id: 66554e51fda461267ce7ad82
paper_authors.name: Sirui Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:32:10.673Z
paper_authors._id: 654b3971b3f4b6f747e590a0
paper_authors.avatarUrl: /avatars/103fd588070a306cf624e27a678cd0d5.svg
paper_authors.fullname: Sirui Xie
paper_authors.type: user
paper_authors.user: SiriusXie
paper_authors._id: 66554e51fda461267ce7ad83
paper_authors.name: Zhisheng Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:32:19.947Z
paper_authors._id: 630846e82d33960e6b7a43ce
paper_authors.avatarUrl: /avatars/058ad9a17d51ce9e1952d2677b0a47bc.svg
paper_authors.fullname: Xiao
paper_authors.type: user
paper_authors.user: Zhisheng
paper_authors._id: 66554e51fda461267ce7ad84
paper_authors.name: Diederik P Kingma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:32:25.690Z
paper_authors._id: 630874fd6fb2ea4413f0dd78
paper_authors.avatarUrl: /avatars/aac7fe259f341655b34a3381ee197226.svg
paper_authors.fullname: Diederik P Kingma
paper_authors.type: user
paper_authors.user: dpkingma
paper_authors._id: 66554e51fda461267ce7ad85
paper_authors.name: Tingbo Hou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:32:34.034Z
paper_authors._id: 655846d7ed8df83128f5826a
paper_authors.avatarUrl: /avatars/d7ce174d7d1b8614d5f6f071225c0057.svg
paper_authors.fullname: Hou
paper_authors.type: user
paper_authors.user: Tingbo
paper_authors._id: 66554e51fda461267ce7ad86
paper_authors.name: Ying Nian Wu
paper_authors._id: 66554e51fda461267ce7ad87
paper_authors.name: Kevin Patrick Murphy
paper_authors._id: 66554e51fda461267ce7ad88
paper_authors.name: Tim Salimans
paper_authors._id: 66554e51fda461267ce7ad89
paper_authors.name: Ben Poole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:33:19.776Z
paper_authors._id: 63213768d2d45f315183abc3
paper_authors.avatarUrl: /avatars/521f52f430b73e514afb06e3a9d6779d.svg
paper_authors.fullname: Ben Poole
paper_authors.type: user
paper_authors.user: doinkda
paper_authors._id: 66554e51fda461267ce7ad8a
paper_authors.name: Ruiqi Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:33:30.403Z
paper_authors._id: 6137d46d1c0a1ca53676925f
paper_authors.avatarUrl: /avatars/f3d884a3971a858adc49e0f3b1932078.svg
paper_authors.fullname: Gao
paper_authors.type: user
paper_authors.user: Ruiqi
paper_title: EM Distillation for One-step Diffusion Models
paper_summary: While diffusion models can learn complex distributions, sampling requires a
  computationally expensive iterative process. Existing distillation methods
  enable efficient sampling, but have notable limitations, such as performance
  degradation with very few sampling steps, reliance on training data access, or
  mode-seeking optimization that may fail to capture the full distribution. We
  propose EM Distillation (EMD), a maximum likelihood-based approach that
  distills a diffusion model to a one-step generator model with minimal loss of
  perceptual quality. Our approach is derived through the lens of
  Expectation-Maximization (EM), where the generator parameters are updated using
  samples from the joint distribution of the diffusion teacher prior and inferred
  generator latents. We develop a reparametrized sampling scheme and a noise
  cancellation technique that together stabilizes the distillation process. We
  further reveal an interesting connection of our method with existing methods
  that minimize mode-seeking KL. EMD outperforms existing one-step generative
  methods in terms of FID scores on ImageNet-64 and ImageNet-128, and compares
  favorably with prior work on distilling text-to-image diffusion models.
paper_ai_keywords: diffusion models sampling distillation methods maximum likelihood-based approach Expectation-Maximization (EM) joint distribution reparametrized sampling scheme noise cancellation technique mode-seeking KL one-step generative methods FID scores ImageNet-64 ImageNet-128 text-to-image diffusion models