Inductive Moment Matching
Diffusion models and Flow Matching generate high-quality samples but are slow
at inference, and distilling them into few-step models often leads to
instability and extensive tuning. To resolve these trade-offs, we propose
Inductive Moment Matching (IMM), a new class of generative models for one- or
few-step sampling with a single-stage training procedure. Unlike distillation,
IMM does not require pre-training initialization and optimization of two
networks; and unlike Consistency Models, IMM guarantees distribution-level
convergence and remains stable under various hyperparameters and standard model
architectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FID
using only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98
on CIFAR-10 for a model trained from scratch.
67cfa0532c3c436eccedab57
Linqi Zhou
claimed_verified
2025-03-12T08:42:39.700Z
64554c3c9d37c3fb332975ed
/avatars/13001d48b56cca99471a7e75116b70cf.svg
Linqi Zhou
user
alexzhou907
67cfa0532c3c436eccedab58
Stefano Ermon
67cfa0532c3c436eccedab59
Jiaming Song
Inductive Moment Matching
Diffusion models and Flow Matching generate high-quality samples but are slow
at inference, and distilling them into few-step models often leads to
instability and extensive tuning. To resolve these trade-offs, we propose
Inductive Moment Matching (IMM), a new class of generative models for one- or
few-step sampling with a single-stage training procedure. Unlike distillation,
IMM does not require pre-training initialization and optimization of two
networks; and unlike Consistency Models, IMM guarantees distribution-level
convergence and remains stable under various hyperparameters and standard model
architectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FID
using only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98
on CIFAR-10 for a model trained from scratch.
Inductive Moment Matching (IMM) is introduced as a generative model for efficient one- or few-step sampling, outperforming diffusion models in terms of quality and speed.
diffusion models Flow Matching Inductive Moment Matching IMM ImageNet-256x256 FID CIFAR-10 distribution-level convergence consistency models