title: Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply
    Better Samples
summary: Although diffusion models can generate remarkably high-quality samples, they
  are intrinsically bottlenecked by their expensive iterative sampling procedure.
  Consistency models (CMs) have recently emerged as a promising diffusion model
  distillation method, reducing the cost of sampling by generating high-fidelity
  samples in just a few iterations. Consistency model distillation aims to solve
  the probability flow ordinary differential equation (ODE) defined by an
  existing diffusion model. CMs are not directly trained to minimize error
  against an ODE solver, rather they use a more computationally tractable
  objective. As a way to study how effectively CMs solve the probability flow
  ODE, and the effect that any induced error has on the quality of generated
  samples, we introduce Direct CMs, which directly minimize this error.
  Intriguingly, we find that Direct CMs reduce the ODE solving error compared to
  CMs but also result in significantly worse sample quality, calling into
  question why exactly CMs work well in the first place. Full code is available
  at: https://github.com/layer6ai-labs/direct-cms.
paper_authors._id: 67377372c87fc7a2bf0e0ec1
paper_authors.name: Noël Vouitsis
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T17:25:40.635Z
paper_authors._id: 6435e521a4bd75c62cc0a64c
paper_authors.avatarUrl: /avatars/50a3860e18ed8347489431b749d4cee3.svg
paper_authors.fullname: Noel Vouitsis
paper_authors.type: user
paper_authors.user: NoelVouitsis
paper_authors._id: 67377372c87fc7a2bf0e0ec2
paper_authors.name: Rasa Hosseinzadeh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:07:29.887Z
paper_authors._id: 631109dbfa95534e2189dd9c
paper_authors.avatarUrl: /avatars/a852580d729600bd26d21ff45ada45eb.svg
paper_authors.fullname: Rasa Hosseinzadeh
paper_authors.type: user
paper_authors.user: rasaHusen
paper_authors._id: 67377372c87fc7a2bf0e0ec3
paper_authors.name: Brendan Leigh Ross
paper_authors._id: 67377372c87fc7a2bf0e0ec4
paper_authors.name: Valentin Villecroze
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:07:41.786Z
paper_authors._id: 651ed211b54adab2d433346a
paper_authors.avatarUrl: /avatars/d2bcdd2091d8efd984e99e915c5d23dc.svg
paper_authors.fullname: Valentin Villecroze
paper_authors.type: user
paper_authors.user: UnrealLink
paper_authors._id: 67377372c87fc7a2bf0e0ec5
paper_authors.name: Satya Krishna Gorti
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:07:48.291Z
paper_authors._id: 65bff4266a6b6de5e4193c30
paper_authors.avatarUrl: /avatars/3ccee2824d36700edf840a6b83812a08.svg
paper_authors.fullname: Satya Krishna Gorti
paper_authors.type: user
paper_authors.user: satyakrishnagorti
paper_authors._id: 67377372c87fc7a2bf0e0ec6
paper_authors.name: Jesse C. Cresswell
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:07:56.657Z
paper_authors._id: 65adf6fbae773a8112f4b9ad
paper_authors.avatarUrl: /avatars/38d2eb81bf1f9b8c4a1903d21c48f365.svg
paper_authors.fullname: Jesse Cresswell
paper_authors.type: user
paper_authors.user: JesseCresswell
paper_authors._id: 67377372c87fc7a2bf0e0ec7
paper_authors.name: Gabriel Loaiza-Ganem
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:08:01.813Z
paper_authors._id: 6324d5b67d6d0cbbe2c7e43c
paper_authors.avatarUrl: /avatars/fc97789a9cbc36ea752b1a1ae98b36d2.svg
paper_authors.fullname: Gabriel Loaiza-Ganem
paper_authors.type: user
paper_authors.user: gabloa
paper_title: Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply
    Better Samples
paper_summary: Although diffusion models can generate remarkably high-quality samples, they
  are intrinsically bottlenecked by their expensive iterative sampling procedure.
  Consistency models (CMs) have recently emerged as a promising diffusion model
  distillation method, reducing the cost of sampling by generating high-fidelity
  samples in just a few iterations. Consistency model distillation aims to solve
  the probability flow ordinary differential equation (ODE) defined by an
  existing diffusion model. CMs are not directly trained to minimize error
  against an ODE solver, rather they use a more computationally tractable
  objective. As a way to study how effectively CMs solve the probability flow
  ODE, and the effect that any induced error has on the quality of generated
  samples, we introduce Direct CMs, which directly minimize this error.
  Intriguingly, we find that Direct CMs reduce the ODE solving error compared to
  CMs but also result in significantly worse sample quality, calling into
  question why exactly CMs work well in the first place. Full code is available
  at: https://github.com/layer6ai-labs/direct-cms.
paper_ai_keywords: diffusion models consistency models (CMs) diffusion model distillation probability flow ordinary differential equation (ODE) Direct CMs