title: LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
summary: Latent Consistency Models (LCMs) have achieved impressive performance in
  accelerating text-to-image generative tasks, producing high-quality images with
  minimal inference steps. LCMs are distilled from pre-trained latent diffusion
  models (LDMs), requiring only ~32 A100 GPU training hours. This report further
  extends LCMs' potential in two aspects: First, by applying LoRA distillation to
  Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded
  LCM's scope to larger models with significantly less memory consumption,
  achieving superior image generation quality. Second, we identify the LoRA
  parameters obtained through LCM distillation as a universal Stable-Diffusion
  acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into
  various Stable-Diffusion fine-tuned models or LoRAs without training, thus
  representing a universally applicable accelerator for diverse image generation
  tasks. Compared with previous numerical PF-ODE solvers such as DDIM,
  DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that
  possesses strong generalization abilities. Project page:
  https://github.com/luosiallen/latent-consistency-model.
paper_authors._id: 654d9b69f714720ae27e332e
paper_authors.name: Simian Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T08:55:54.016Z
paper_authors._id: 652a4f174d0eef4a2bf85096
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ofYN04WWtD60oeDP45pQN.jpeg
paper_authors.fullname: Simian Luo
paper_authors.type: user
paper_authors.user: SimianLuo
paper_authors._id: 654d9b69f714720ae27e332f
paper_authors.name: Yiqin Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T08:56:02.144Z
paper_authors._id: 65237947706c755148c2678b
paper_authors.avatarUrl: /avatars/b7ba5dbf6286e8fa8f4f4999a473bd94.svg
paper_authors.fullname: Yiqin Tan
paper_authors.type: user
paper_authors.user: tyq1024
paper_authors._id: 654d9b69f714720ae27e3330
paper_authors.name: Suraj Patil
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T09:02:32.127Z
paper_authors._id: 5ec0135ded25d76864d553f1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1600014820272-5ec0135ded25d76864d553f1.jpeg
paper_authors.fullname: Suraj Patil
paper_authors.type: user
paper_authors.user: valhalla
paper_authors._id: 654d9b69f714720ae27e3331
paper_authors.name: Daniel Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T09:03:10.095Z
paper_authors._id: 64433881f99d6629a71d39ed
paper_authors.avatarUrl: /avatars/35e7257e357064e77cfdcae384afe36f.svg
paper_authors.fullname: Daniel Gu
paper_authors.type: user
paper_authors.user: dg845
paper_authors._id: 654d9b69f714720ae27e3332
paper_authors.name: Patrick von Platen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T08:57:29.064Z
paper_authors._id: 5dfcb1aada6d0311fd3d5448
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg
paper_authors.fullname: Patrick von Platen
paper_authors.type: user
paper_authors.user: patrickvonplaten
paper_authors._id: 654d9b69f714720ae27e3333
paper_authors.name: Apolinário Passos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T09:01:55.660Z
paper_authors._id: 624bebf604abc7ebb01789af
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1649143001781-624bebf604abc7ebb01789af.jpeg
paper_authors.fullname: Apolinário from multimodal AI art
paper_authors.type: user
paper_authors.user: multimodalart
paper_authors._id: 654d9b69f714720ae27e3334
paper_authors.name: Longbo Huang
paper_authors._id: 654d9b69f714720ae27e3335
paper_authors.name: Jian Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T09:00:08.819Z
paper_authors._id: 63b5c13a889aa6707f0e8548
paper_authors.avatarUrl: /avatars/a475b141fdc859967a9b63252620be05.svg
paper_authors.fullname: Jian Li
paper_authors.type: user
paper_authors.user: jianl
paper_authors._id: 654d9b69f714720ae27e3336
paper_authors.name: Hang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-10T09:00:17.152Z
paper_authors._id: 63bc8467b8c61b8aa49c3e11
paper_authors.avatarUrl: /avatars/f75a33b6bba1bb6aecbe018dcef39131.svg
paper_authors.fullname: Hang Zhao
paper_authors.type: user
paper_authors.user: sgzhaohang
paper_title: LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
paper_summary: Latent Consistency Models (LCMs) have achieved impressive performance in
  accelerating text-to-image generative tasks, producing high-quality images with
  minimal inference steps. LCMs are distilled from pre-trained latent diffusion
  models (LDMs), requiring only ~32 A100 GPU training hours. This report further
  extends LCMs' potential in two aspects: First, by applying LoRA distillation to
  Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded
  LCM's scope to larger models with significantly less memory consumption,
  achieving superior image generation quality. Second, we identify the LoRA
  parameters obtained through LCM distillation as a universal Stable-Diffusion
  acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into
  various Stable-Diffusion fine-tuned models or LoRAs without training, thus
  representing a universally applicable accelerator for diverse image generation
  tasks. Compared with previous numerical PF-ODE solvers such as DDIM,
  DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that
  possesses strong generalization abilities. Project page:
  https://github.com/luosiallen/latent-consistency-model.
paper_ai_keywords: Latent Consistency Models (LCMs) text-to-image generative tasks high-quality images minimal inference steps latent diffusion models (LDMs) LoRA distillation Stable-Diffusion models SD-V1.5 SSD-1B SDXL memory consumption image generation quality LCM-LoRA Stable-Diffusion fine-tuned models neural PF-ODE solver DDIM DPM-Solver