title: Conditional Diffusion Distillation
summary: Generative diffusion models provide strong priors for text-to-image
  generation and thereby serve as a foundation for conditional generation tasks
  such as image editing, restoration, and super-resolution. However, one major
  limitation of diffusion models is their slow sampling time. To address this
  challenge, we present a novel conditional distillation method designed to
  supplement the diffusion priors with the help of image conditions, allowing for
  conditional sampling with very few steps. We directly distill the unconditional
  pre-training in a single stage through joint-learning, largely simplifying the
  previous two-stage procedures that involve both distillation and conditional
  finetuning separately. Furthermore, our method enables a new
  parameter-efficient distillation mechanism that distills each task with only a
  small number of additional parameters combined with the shared frozen
  unconditional backbone. Experiments across multiple tasks including
  super-resolution, image editing, and depth-to-image generation demonstrate that
  our method outperforms existing distillation techniques for the same sampling
  time. Notably, our method is the first distillation strategy that can match the
  performance of the much slower fine-tuned conditional diffusion models.
paper_authors._id: 651b94c33265a1bb83a0aac9
paper_authors.name: Kangfu Mei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-03T11:31:56.284Z
paper_authors._id: 635c1f1b37c6a2c12e335604
paper_authors.avatarUrl: /avatars/edc215a20af0ec5be83721fb4abc1ab5.svg
paper_authors.fullname: Kangfu Mei
paper_authors.type: user
paper_authors.user: MKFMIKU
paper_authors._id: 651b94c33265a1bb83a0aaca
paper_authors.name: Mauricio Delbracio
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-03T15:06:35.900Z
paper_authors._id: 651c262b497e4d326780c18a
paper_authors.avatarUrl: /avatars/65d50faaa22c5bbd982b5652a530da2c.svg
paper_authors.fullname: Mauricio Delbracio
paper_authors.type: user
paper_authors.user: mdelbra
paper_authors._id: 651b94c33265a1bb83a0aacb
paper_authors.name: Hossein Talebi
paper_authors._id: 651b94c33265a1bb83a0aacc
paper_authors.name: Zhengzhong Tu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-03T11:32:13.114Z
paper_authors._id: 62548d5fef3debb2ddf91217
paper_authors.avatarUrl: /avatars/14975b45568f9c399c92c3986b6ce83e.svg
paper_authors.fullname: Zhengzhong Tu
paper_authors.type: user
paper_authors.user: vztu
paper_authors._id: 651b94c33265a1bb83a0aacd
paper_authors.name: Vishal M. Patel
paper_authors._id: 651b94c33265a1bb83a0aace
paper_authors.name: Peyman Milanfar
paper_title: Conditional Diffusion Distillation
paper_summary: Generative diffusion models provide strong priors for text-to-image
  generation and thereby serve as a foundation for conditional generation tasks
  such as image editing, restoration, and super-resolution. However, one major
  limitation of diffusion models is their slow sampling time. To address this
  challenge, we present a novel conditional distillation method designed to
  supplement the diffusion priors with the help of image conditions, allowing for
  conditional sampling with very few steps. We directly distill the unconditional
  pre-training in a single stage through joint-learning, largely simplifying the
  previous two-stage procedures that involve both distillation and conditional
  finetuning separately. Furthermore, our method enables a new
  parameter-efficient distillation mechanism that distills each task with only a
  small number of additional parameters combined with the shared frozen
  unconditional backbone. Experiments across multiple tasks including
  super-resolution, image editing, and depth-to-image generation demonstrate that
  our method outperforms existing distillation techniques for the same sampling
  time. Notably, our method is the first distillation strategy that can match the
  performance of the much slower fine-tuned conditional diffusion models.
paper_ai_keywords: generative diffusion models text-to-image generation conditional generation image editing restoration slow sampling time conditional distillation unconditional pre-training joint-learning parameter-efficient distillation frozen unconditional backbone depth-to-image generation