title: Common Diffusion Noise Schedules and Sample Steps are Flawed
summary: We discover that common diffusion noise schedules do not enforce the last
  timestep to have zero signal-to-noise ratio (SNR), and some implementations of
  diffusion samplers do not start from the last timestep. Such designs are flawed
  and do not reflect the fact that the model is given pure Gaussian noise at
  inference, creating a discrepancy between training and inference. We show that
  the flawed design causes real problems in existing implementations. In Stable
  Diffusion, it severely limits the model to only generate images with medium
  brightness and prevents it from generating very bright and dark samples. We
  propose a few simple fixes: (1) rescale the noise schedule to enforce zero
  terminal SNR; (2) train the model with v prediction; (3) change the sampler to
  always start from the last timestep; (4) rescale classifier-free guidance to
  prevent over-exposure. These simple changes ensure the diffusion process is
  congruent between training and inference and allow the model to generate
  samples more faithful to the original data distribution.
paper_authors._id: 64645c72884f2e3e1ced059f
paper_authors.name: Shanchuan Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T08:03:40.137Z
paper_authors._id: 645863f7dc18eb1a9b5d29df
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645863f7dc18eb1a9b5d29df/t49Nnyl4tbkUn7CmQqKZh.jpeg
paper_authors.fullname: Peter Lin
paper_authors.type: user
paper_authors.user: PeterL1n
paper_authors._id: 64645c72884f2e3e1ced05a0
paper_authors.name: Bingchen Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T22:34:38.860Z
paper_authors._id: 62f9e1f90697d2242198c1db
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660543471146-noauth.jpeg
paper_authors.fullname: bingchen liu
paper_authors.type: user
paper_authors.user: codegeasslbc
paper_authors._id: 64645c72884f2e3e1ced05a1
paper_authors.name: Jiashi Li
paper_authors._id: 64645c72884f2e3e1ced05a2
paper_authors.name: Xiao Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T08:03:28.428Z
paper_authors._id: 62b3d2490385d6048985b42b
paper_authors.avatarUrl: /avatars/1c4e534d086d53ef1057fbc00334350a.svg
paper_authors.fullname: Xiao Yang
paper_authors.type: user
paper_authors.user: seravee008
paper_title: Common Diffusion Noise Schedules and Sample Steps are Flawed
paper_summary: We discover that common diffusion noise schedules do not enforce the last
  timestep to have zero signal-to-noise ratio (SNR), and some implementations of
  diffusion samplers do not start from the last timestep. Such designs are flawed
  and do not reflect the fact that the model is given pure Gaussian noise at
  inference, creating a discrepancy between training and inference. We show that
  the flawed design causes real problems in existing implementations. In Stable
  Diffusion, it severely limits the model to only generate images with medium
  brightness and prevents it from generating very bright and dark samples. We
  propose a few simple fixes: (1) rescale the noise schedule to enforce zero
  terminal SNR; (2) train the model with v prediction; (3) change the sampler to
  always start from the last timestep; (4) rescale classifier-free guidance to
  prevent over-exposure. These simple changes ensure the diffusion process is
  congruent between training and inference and allow the model to generate
  samples more faithful to the original data distribution.
paper_ai_keywords: diffusion noise schedules signal-to-noise ratio (SNR) diffusion samplers Gaussian noise training inference discrepancy Stable Diffusion image generation v prediction classifier-free guidance over-exposure diffusion process data distribution