Table 1 also shows the codelengths of our CIFAR10 models. The gap between train and test is at
most 0.03 bits per dimension, which is comparable to the gaps reported with other likelihood-based
models and indicates that our diffusion model is not overﬁtting (see Appendix D for nearest neighbor
visualizations). Still, while our lossless codelengths are better than the large estimates reported for
energy based models and score matching using annealed importance sampling [11], they are not
competitive with other types of likelihood-based generative models [7].
Since our samples are nonetheless of high quality, we conclude that diffusion models have an inductive
bias that makes them excellent lossy compressors. Treating the variational bound terms L1 +··· + LT
as rate and L0 as distortion, our CIFAR10 model with the highest quality samples has a rate of 1.78
bits/dim and a distortion of 1.97 bits/dim, which amounts to a root mean squared error of 0.95 on a
Denoising Diffusion Probabilistic Models