title: Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
summary: Consistency models (CMs) are a powerful class of diffusion-based generative
  models optimized for fast sampling. Most existing CMs are trained using
  discretized timesteps, which introduce additional hyperparameters and are prone
  to discretization errors. While continuous-time formulations can mitigate these
  issues, their success has been limited by training instability. To address
  this, we propose a simplified theoretical framework that unifies previous
  parameterizations of diffusion models and CMs, identifying the root causes of
  instability. Based on this analysis, we introduce key improvements in diffusion
  process parameterization, network architecture, and training objectives. These
  changes enable us to train continuous-time CMs at an unprecedented scale,
  reaching 1.5B parameters on ImageNet 512x512. Our proposed training algorithm,
  using only two sampling steps, achieves FID scores of 2.06 on CIFAR-10, 1.48 on
  ImageNet 64x64, and 1.88 on ImageNet 512x512, narrowing the gap in FID scores
  with the best existing diffusion models to within 10%.
paper_authors._id: 670ff433d126c976117487b3
paper_authors.name: Cheng Lu
paper_authors._id: 670ff433d126c976117487b4
paper_authors.name: Yang Song
paper_title: Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
paper_summary: Consistency models (CMs) are a powerful class of diffusion-based generative
  models optimized for fast sampling. Most existing CMs are trained using
  discretized timesteps, which introduce additional hyperparameters and are prone
  to discretization errors. While continuous-time formulations can mitigate these
  issues, their success has been limited by training instability. To address
  this, we propose a simplified theoretical framework that unifies previous
  parameterizations of diffusion models and CMs, identifying the root causes of
  instability. Based on this analysis, we introduce key improvements in diffusion
  process parameterization, network architecture, and training objectives. These
  changes enable us to train continuous-time CMs at an unprecedented scale,
  reaching 1.5B parameters on ImageNet 512x512. Our proposed training algorithm,
  using only two sampling steps, achieves FID scores of 2.06 on CIFAR-10, 1.48 on
  ImageNet 64x64, and 1.88 on ImageNet 512x512, narrowing the gap in FID scores
  with the best existing diffusion models to within 10%.
paper_ai_summary: A new theoretical framework and algorithm improve continuous-time consistency models, enabling large-scale training and achieving near-state-of-the-art FID scores.
paper_ai_keywords: consistency models diffusion-based generative models discretized timesteps continuous-time formulations training instability diffusion process parameterization network architecture training objectives ImageNet CIFAR-10 FID scores