title: DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model
    Statistics
summary: Diffusion probabilistic models (DPMs) have exhibited excellent performance
  for high-fidelity image generation while suffering from inefficient sampling.
  Recent works accelerate the sampling procedure by proposing fast ODE solvers
  that leverage the specific ODE form of DPMs. However, they highly rely on
  specific parameterization during inference (such as noise/data prediction),
  which might not be the optimal choice. In this work, we propose a novel
  formulation towards the optimal parameterization during sampling that minimizes
  the first-order discretization error of the ODE solution. Based on such
  formulation, we propose DPM-Solver-v3, a new fast ODE solver for DPMs
  by introducing several coefficients efficiently computed on the pretrained
  model, which we call empirical model statistics. We further
  incorporate multistep methods and a predictor-corrector framework, and propose
  some techniques for improving sample quality at small numbers of function
  evaluations (NFE) or large guidance scales. Experiments show that DPM-Solver-v3
  achieves consistently better or comparable performance in both unconditional
  and conditional sampling with both pixel-space and latent-space DPMs,
  especially in 5sim10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE)
  on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable
  Diffusion, bringing a speed-up of 15\%sim30\% compared to previous
  state-of-the-art training-free methods. Code is available at
  https://github.com/thu-ml/DPM-Solver-v3.
paper_authors._id: 65360aaa910b844786c84f3f
paper_authors.name: Kaiwen Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-23T08:32:21.989Z
paper_authors._id: 652bf7edc3cba555d5673c6e
paper_authors.avatarUrl: /avatars/78f6416c30203b30671f8423f061c657.svg
paper_authors.fullname: Kaiwen Zheng
paper_authors.type: user
paper_authors.user: worstcoder
paper_authors._id: 65360aaa910b844786c84f40
paper_authors.name: Cheng Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-23T08:42:12.591Z
paper_authors._id: 6308ab64e3246e2be8884f4f
paper_authors.avatarUrl: /avatars/7d8002caeacf27258996f31e4a2e186f.svg
paper_authors.fullname: LuChengTHU
paper_authors.type: user
paper_authors.user: LuChengTHU
paper_authors._id: 65360aaa910b844786c84f41
paper_authors.name: Jianfei Chen
paper_authors._id: 65360aaa910b844786c84f42
paper_authors.name: Jun Zhu
paper_title: DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model
    Statistics
paper_summary: Diffusion probabilistic models (DPMs) have exhibited excellent performance
  for high-fidelity image generation while suffering from inefficient sampling.
  Recent works accelerate the sampling procedure by proposing fast ODE solvers
  that leverage the specific ODE form of DPMs. However, they highly rely on
  specific parameterization during inference (such as noise/data prediction),
  which might not be the optimal choice. In this work, we propose a novel
  formulation towards the optimal parameterization during sampling that minimizes
  the first-order discretization error of the ODE solution. Based on such
  formulation, we propose DPM-Solver-v3, a new fast ODE solver for DPMs
  by introducing several coefficients efficiently computed on the pretrained
  model, which we call empirical model statistics. We further
  incorporate multistep methods and a predictor-corrector framework, and propose
  some techniques for improving sample quality at small numbers of function
  evaluations (NFE) or large guidance scales. Experiments show that DPM-Solver-v3
  achieves consistently better or comparable performance in both unconditional
  and conditional sampling with both pixel-space and latent-space DPMs,
  especially in 5sim10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE)
  on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable
  Diffusion, bringing a speed-up of 15\%sim30\% compared to previous
  state-of-the-art training-free methods. Code is available at
  https://github.com/thu-ml/DPM-Solver-v3.
paper_ai_summary: A novel ODE solver for diffusion probabilistic models optimizes sampling efficiency and sample quality by minimizing discretization errors and introducing empirical model statistics, achieving faster and better results in both pixel-space and latent-space models.
paper_ai_keywords: diffusion probabilistic models DPMs fast ODE solvers DPM-Solver-v3 empirical model statistics multistep methods predictor-corrector framework unconditional sampling conditional sampling pixel-space latent-space FIDs MSE Stable Diffusion