title: MLCM: Multistep Consistency Distillation of Latent Diffusion Model
summary: Distilling large latent diffusion models (LDMs) into ones that are fast to
  sample from is attracting growing research interest. However, the majority of
  existing methods face a dilemma where they either (i) depend on multiple
  individual distilled models for different sampling budgets, or (ii) sacrifice
  generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8)
  sampling steps. To address these, we extend the recent multistep consistency
  distillation (MCD) strategy to representative LDMs, establishing the Multistep
  Latent Consistency Models (MLCMs) approach for low-cost high-quality image
  synthesis. MLCM serves as a unified model for various sampling steps due to the
  promise of MCD. We further augment MCD with a progressive training strategy to
  strengthen inter-segment consistency to boost the quality of few-step
  generations. We take the states from the sampling trajectories of the teacher
  model as training data for MLCMs to lift the requirements for high-quality
  training datasets and to bridge the gap between the training and inference of
  the distilled model. MLCM is compatible with preference learning strategies for
  further improvement of visual quality and aesthetic appeal. Empirically, MLCM
  can generate high-quality, delightful images with only 2-8 sampling steps. On
  the MSCOCO-2017 5K benchmark, MLCM distilled from SDXL gets a CLIP Score of
  33.30, Aesthetic Score of 6.19, and Image Reward of 1.20 with only 4 steps,
  substantially surpassing 4-step LCM [23], 8-step SDXL-Lightning [17], and
  8-step HyperSD [33]. We also demonstrate the versatility of MLCMs in
  applications including controllable generation, image style transfer, and
  Chinese-to-image generation.
paper_authors._id: 6667c2e82cef2f5b6d8291a4
paper_authors.name: Qingsong Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T10:43:15.366Z
paper_authors._id: 64184911db24526c7c9ca035
paper_authors.avatarUrl: /avatars/2be15485bdd9e00857d4bff107d1577e.svg
paper_authors.fullname: qingsong xie
paper_authors.type: user
paper_authors.user: kongming11
paper_authors._id: 6667c2e82cef2f5b6d8291a5
paper_authors.name: Zhenyi Liao
paper_authors._id: 6667c2e82cef2f5b6d8291a6
paper_authors.name: Zhijie Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T10:43:28.705Z
paper_authors._id: 64bba541da140e461924dfed
paper_authors.avatarUrl: /avatars/367993765b0ca3734b2b100db33ed787.svg
paper_authors.fullname: zhijie deng
paper_authors.type: user
paper_authors.user: zhijie3
paper_authors._id: 6667c2e82cef2f5b6d8291a7
paper_authors.name: Chen chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-11T13:26:42.883Z
paper_authors._id: 649d47e06503630c5269d93a
paper_authors.avatarUrl: /avatars/6812c9f79f2bd753ed68afb03baac202.svg
paper_authors.fullname: Chen Chen
paper_authors.type: user
paper_authors.user: beckhamchen
paper_authors._id: 6667c2e82cef2f5b6d8291a8
paper_authors.name: Shixiang Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T10:43:59.396Z
paper_authors._id: 6436403bf3b08e267d9f0329
paper_authors.avatarUrl: /avatars/a5d9c3d47073e71e4cea124d9c17356d.svg
paper_authors.fullname: SHIXIANG TANG
paper_authors.type: user
paper_authors.user: tangshixiang
paper_authors._id: 6667c2e82cef2f5b6d8291a9
paper_authors.name: Haonan Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T10:44:05.156Z
paper_authors._id: 603864181b642aef9350af81
paper_authors.avatarUrl: /avatars/27ec0b885cbedef007876c8c98c69d15.svg
paper_authors.fullname: Haonan Lu
paper_authors.type: user
paper_authors.user: luhaonan20
paper_title: MLCM: Multistep Consistency Distillation of Latent Diffusion Model
paper_summary: Distilling large latent diffusion models (LDMs) into ones that are fast to
  sample from is attracting growing research interest. However, the majority of
  existing methods face a dilemma where they either (i) depend on multiple
  individual distilled models for different sampling budgets, or (ii) sacrifice
  generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8)
  sampling steps. To address these, we extend the recent multistep consistency
  distillation (MCD) strategy to representative LDMs, establishing the Multistep
  Latent Consistency Models (MLCMs) approach for low-cost high-quality image
  synthesis. MLCM serves as a unified model for various sampling steps due to the
  promise of MCD. We further augment MCD with a progressive training strategy to
  strengthen inter-segment consistency to boost the quality of few-step
  generations. We take the states from the sampling trajectories of the teacher
  model as training data for MLCMs to lift the requirements for high-quality
  training datasets and to bridge the gap between the training and inference of
  the distilled model. MLCM is compatible with preference learning strategies for
  further improvement of visual quality and aesthetic appeal. Empirically, MLCM
  can generate high-quality, delightful images with only 2-8 sampling steps. On
  the MSCOCO-2017 5K benchmark, MLCM distilled from SDXL gets a CLIP Score of
  33.30, Aesthetic Score of 6.19, and Image Reward of 1.20 with only 4 steps,
  substantially surpassing 4-step LCM [23], 8-step SDXL-Lightning [17], and
  8-step HyperSD [33]. We also demonstrate the versatility of MLCMs in
  applications including controllable generation, image style transfer, and
  Chinese-to-image generation.
paper_ai_summary: A new approach, Multistep Latent Consistency Models (MLCMs), extends the multistep consistency distillation strategy to latent diffusion models, enabling high-quality image synthesis with fewer sampling steps.
paper_ai_keywords: multistep consistency distillation Multistep Latent Consistency Models progressive training sampling trajectories preference learning CLIP Score Aesthetic Score Image Reward controllable generation image style transfer Chinese-to-image generation