title: Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
summary: Diffusion models (DMs) have established themselves as the state-of-the-art
  generative modeling approach in the visual domain and beyond. A crucial
  drawback of DMs is their slow sampling speed, relying on many sequential
  function evaluations through large neural networks. Sampling from DMs can be
  seen as solving a differential equation through a discretized set of noise
  levels known as the sampling schedule. While past works primarily focused on
  deriving efficient solvers, little attention has been given to finding optimal
  sampling schedules, and the entire literature relies on hand-crafted
  heuristics. In this work, for the first time, we propose a general and
  principled approach to optimizing the sampling schedules of DMs for
  high-quality outputs, called Align Your Steps. We leverage methods
  from stochastic calculus and find optimal schedules specific to different
  solvers, trained DMs and datasets. We evaluate our novel approach on several
  image, video as well as 2D toy data synthesis benchmarks, using a variety of
  different samplers, and observe that our optimized schedules outperform
  previous hand-crafted schedules in almost all experiments. Our method
  demonstrates the untapped potential of sampling schedule optimization,
  especially in the few-step synthesis regime.
paper_authors._id: 6628631f09614385712460dd
paper_authors.name: Amirmojtaba Sabour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-24T12:28:06.623Z
paper_authors._id: 656015f28138827138c70858
paper_authors.avatarUrl: /avatars/b8471ae4d80f078c7c928fc3d8f49126.svg
paper_authors.fullname: Amirmojtaba Sabour
paper_authors.type: user
paper_authors.user: amsabour
paper_authors._id: 6628631f09614385712460de
paper_authors.name: Sanja Fidler
paper_authors._id: 6628631f09614385712460df
paper_authors.name: Karsten Kreis
paper_title: Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
paper_summary: Diffusion models (DMs) have established themselves as the state-of-the-art
  generative modeling approach in the visual domain and beyond. A crucial
  drawback of DMs is their slow sampling speed, relying on many sequential
  function evaluations through large neural networks. Sampling from DMs can be
  seen as solving a differential equation through a discretized set of noise
  levels known as the sampling schedule. While past works primarily focused on
  deriving efficient solvers, little attention has been given to finding optimal
  sampling schedules, and the entire literature relies on hand-crafted
  heuristics. In this work, for the first time, we propose a general and
  principled approach to optimizing the sampling schedules of DMs for
  high-quality outputs, called Align Your Steps. We leverage methods
  from stochastic calculus and find optimal schedules specific to different
  solvers, trained DMs and datasets. We evaluate our novel approach on several
  image, video as well as 2D toy data synthesis benchmarks, using a variety of
  different samplers, and observe that our optimized schedules outperform
  previous hand-crafted schedules in almost all experiments. Our method
  demonstrates the untapped potential of sampling schedule optimization,
  especially in the few-step synthesis regime.
paper_ai_keywords: diffusion models sampling speed function evaluations differential equation noise levels sampling schedule efficient solvers stochastic calculus sampling schedule optimization high-quality outputs Align Your Steps image synthesis video synthesis 2D toy data synthesis few-step synthesis regime