Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply
  Better Samples
Although diffusion models can generate remarkably high-quality samples, they
are intrinsically bottlenecked by their expensive iterative sampling procedure.
Consistency models (CMs) have recently emerged as a promising diffusion model
distillation method, reducing the cost of sampling by generating high-fidelity
samples in just a few iterations. Consistency model distillation aims to solve
the probability flow ordinary differential equation (ODE) defined by an
existing diffusion model. CMs are not directly trained to minimize error
against an ODE solver, rather they use a more computationally tractable
objective. As a way to study how effectively CMs solve the probability flow
ODE, and the effect that any induced error has on the quality of generated
samples, we introduce Direct CMs, which directly minimize this error.
Intriguingly, we find that Direct CMs reduce the ODE solving error compared to
CMs but also result in significantly worse sample quality, calling into
question why exactly CMs work well in the first place. Full code is available
at: https://github.com/layer6ai-labs/direct-cms.
67377372c87fc7a2bf0e0ec1
Noël Vouitsis
claimed_verified
2024-11-15T17:25:40.635Z
6435e521a4bd75c62cc0a64c
/avatars/50a3860e18ed8347489431b749d4cee3.svg
Noel Vouitsis
user
NoelVouitsis
67377372c87fc7a2bf0e0ec2
Rasa Hosseinzadeh
admin_assigned
2024-11-18T08:07:29.887Z
631109dbfa95534e2189dd9c
/avatars/a852580d729600bd26d21ff45ada45eb.svg
Rasa Hosseinzadeh
user
rasaHusen
67377372c87fc7a2bf0e0ec3
Brendan Leigh Ross
67377372c87fc7a2bf0e0ec4
Valentin Villecroze
admin_assigned
2024-11-18T08:07:41.786Z
651ed211b54adab2d433346a
/avatars/d2bcdd2091d8efd984e99e915c5d23dc.svg
Valentin Villecroze
user
UnrealLink
67377372c87fc7a2bf0e0ec5
Satya Krishna Gorti
admin_assigned
2024-11-18T08:07:48.291Z
65bff4266a6b6de5e4193c30
/avatars/3ccee2824d36700edf840a6b83812a08.svg
Satya Krishna Gorti
user
satyakrishnagorti
67377372c87fc7a2bf0e0ec6
Jesse C. Cresswell
admin_assigned
2024-11-18T08:07:56.657Z
65adf6fbae773a8112f4b9ad
/avatars/38d2eb81bf1f9b8c4a1903d21c48f365.svg
Jesse Cresswell
user
JesseCresswell
67377372c87fc7a2bf0e0ec7
Gabriel Loaiza-Ganem
admin_assigned
2024-11-18T08:08:01.813Z
6324d5b67d6d0cbbe2c7e43c
/avatars/fc97789a9cbc36ea752b1a1ae98b36d2.svg
Gabriel Loaiza-Ganem
user
gabloa
Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply
  Better Samples
Although diffusion models can generate remarkably high-quality samples, they
are intrinsically bottlenecked by their expensive iterative sampling procedure.
Consistency models (CMs) have recently emerged as a promising diffusion model
distillation method, reducing the cost of sampling by generating high-fidelity
samples in just a few iterations. Consistency model distillation aims to solve
the probability flow ordinary differential equation (ODE) defined by an
existing diffusion model. CMs are not directly trained to minimize error
against an ODE solver, rather they use a more computationally tractable
objective. As a way to study how effectively CMs solve the probability flow
ODE, and the effect that any induced error has on the quality of generated
samples, we introduce Direct CMs, which directly minimize this error.
Intriguingly, we find that Direct CMs reduce the ODE solving error compared to
CMs but also result in significantly worse sample quality, calling into
question why exactly CMs work well in the first place. Full code is available
at: https://github.com/layer6ai-labs/direct-cms.
Direct Consistency Models aim to reduce error in solving the probability flow ODE of diffusion models but surprisingly degrade sample quality compared to traditional Consistency Models.
diffusion models consistency models probability flow ordinary differential equation ODE solver sample quality