title: Your Student is Better Than Expected: Adaptive Teacher-Student
    Collaboration for Text-Conditional Diffusion Models
summary: Knowledge distillation methods have recently shown to be a promising
  direction to speedup the synthesis of large-scale diffusion models by requiring
  only a few inference steps. While several powerful distillation methods were
  recently proposed, the overall quality of student samples is typically lower
  compared to the teacher ones, which hinders their practical usage. In this
  work, we investigate the relative quality of samples produced by the teacher
  text-to-image diffusion model and its distilled student version. As our main
  empirical finding, we discover that a noticeable portion of student samples
  exhibit superior fidelity compared to the teacher ones, despite the
  ``approximate'' nature of the student. Based on this finding, we propose an
  adaptive collaboration between student and teacher diffusion models for
  effective text-to-image synthesis. Specifically, the distilled model produces
  the initial sample, and then an oracle decides whether it needs further
  improvements with a slow teacher model. Extensive experiments demonstrate that
  the designed pipeline surpasses state-of-the-art text-to-image alternatives for
  various inference budgets in terms of human preference. Furthermore, the
  proposed approach can be naturally used in popular applications such as
  text-guided image editing and controllable generation.
paper_authors._id: 658139c398aa9fcdd270d6d3
paper_authors.name: Nikita Starodubcev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:12:46.126Z
paper_authors._id: 6410d3a4cfbe9c4400233d1e
paper_authors.avatarUrl: /avatars/9e1c4e0ea5f6964c90013f6e19c41db1.svg
paper_authors.fullname: nikita
paper_authors.type: user
paper_authors.user: quickjkee
paper_authors._id: 658139c398aa9fcdd270d6d4
paper_authors.name: Artem Fedorov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:11:51.167Z
paper_authors._id: 6343fcfad236b24788be85f9
paper_authors.avatarUrl: /avatars/72fe4c2b3aeb3e2ab87cdaf70b6615b8.svg
paper_authors.fullname: Artem Fedorov
paper_authors.type: user
paper_authors.user: Ok0lo
paper_authors._id: 658139c398aa9fcdd270d6d5
paper_authors.name: Artem Babenko
paper_authors._id: 658139c398aa9fcdd270d6d6
paper_authors.name: Dmitry Baranchuk
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:12:01.911Z
paper_authors._id: 62b6cc49752323892323bc04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b6cc49752323892323bc04/gGBld1KJIP9AIpd81L3PC.jpeg
paper_authors.fullname: Dmitry Baranchuk
paper_authors.type: user
paper_authors.user: dbaranchuk
paper_title: Your Student is Better Than Expected: Adaptive Teacher-Student
    Collaboration for Text-Conditional Diffusion Models
paper_summary: Knowledge distillation methods have recently shown to be a promising
  direction to speedup the synthesis of large-scale diffusion models by requiring
  only a few inference steps. While several powerful distillation methods were
  recently proposed, the overall quality of student samples is typically lower
  compared to the teacher ones, which hinders their practical usage. In this
  work, we investigate the relative quality of samples produced by the teacher
  text-to-image diffusion model and its distilled student version. As our main
  empirical finding, we discover that a noticeable portion of student samples
  exhibit superior fidelity compared to the teacher ones, despite the
  ``approximate'' nature of the student. Based on this finding, we propose an
  adaptive collaboration between student and teacher diffusion models for
  effective text-to-image synthesis. Specifically, the distilled model produces
  the initial sample, and then an oracle decides whether it needs further
  improvements with a slow teacher model. Extensive experiments demonstrate that
  the designed pipeline surpasses state-of-the-art text-to-image alternatives for
  various inference budgets in terms of human preference. Furthermore, the
  proposed approach can be naturally used in popular applications such as
  text-guided image editing and controllable generation.
paper_ai_keywords: knowledge distillation diffusion models synthesis text-to-image student samples teacher samples fidelity adaptive collaboration oracle