Conditional Diffusion Distillation
Generative diffusion models provide strong priors for text-to-image
generation and thereby serve as a foundation for conditional generation tasks
such as image editing, restoration, and super-resolution. However, one major
limitation of diffusion models is their slow sampling time. To address this
challenge, we present a novel conditional distillation method designed to
supplement the diffusion priors with the help of image conditions, allowing for
conditional sampling with very few steps. We directly distill the unconditional
pre-training in a single stage through joint-learning, largely simplifying the
previous two-stage procedures that involve both distillation and conditional
finetuning separately. Furthermore, our method enables a new
parameter-efficient distillation mechanism that distills each task with only a
small number of additional parameters combined with the shared frozen
unconditional backbone. Experiments across multiple tasks including
super-resolution, image editing, and depth-to-image generation demonstrate that
our method outperforms existing distillation techniques for the same sampling
time. Notably, our method is the first distillation strategy that can match the
performance of the much slower fine-tuned conditional diffusion models.
651b94c33265a1bb83a0aac9
Kangfu Mei
admin_assigned
2023-10-03T11:31:56.284Z
635c1f1b37c6a2c12e335604
/avatars/edc215a20af0ec5be83721fb4abc1ab5.svg
Kangfu Mei
user
MKFMIKU
651b94c33265a1bb83a0aaca
Mauricio Delbracio
claimed_verified
2023-10-03T15:06:35.900Z
651c262b497e4d326780c18a
/avatars/65d50faaa22c5bbd982b5652a530da2c.svg
Mauricio Delbracio
user
mdelbra
651b94c33265a1bb83a0aacb
Hossein Talebi
651b94c33265a1bb83a0aacc
Zhengzhong Tu
admin_assigned
2023-10-03T11:32:13.114Z
62548d5fef3debb2ddf91217
/avatars/14975b45568f9c399c92c3986b6ce83e.svg
Zhengzhong Tu
user
vztu
651b94c33265a1bb83a0aacd
Vishal M. Patel
651b94c33265a1bb83a0aace
Peyman Milanfar
Conditional Diffusion Distillation
Generative diffusion models provide strong priors for text-to-image
generation and thereby serve as a foundation for conditional generation tasks
such as image editing, restoration, and super-resolution. However, one major
limitation of diffusion models is their slow sampling time. To address this
challenge, we present a novel conditional distillation method designed to
supplement the diffusion priors with the help of image conditions, allowing for
conditional sampling with very few steps. We directly distill the unconditional
pre-training in a single stage through joint-learning, largely simplifying the
previous two-stage procedures that involve both distillation and conditional
finetuning separately. Furthermore, our method enables a new
parameter-efficient distillation mechanism that distills each task with only a
small number of additional parameters combined with the shared frozen
unconditional backbone. Experiments across multiple tasks including
super-resolution, image editing, and depth-to-image generation demonstrate that
our method outperforms existing distillation techniques for the same sampling
time. Notably, our method is the first distillation strategy that can match the
performance of the much slower fine-tuned conditional diffusion models.
A novel single-stage distillation method for generative diffusion models reduces sampling time while maintaining performance across tasks like super-resolution and image editing.
generative diffusion models text-to-image generation conditional generation tasks image editing restoration super-resolution diffusion priors conditional sampling parameter-efficient distillation unconditional pre-training joint-learning fine-tuned conditional diffusion models