C Discussion on related work

Our model architecture, forward process deﬁnition, and prior differ from NCSN [55, 56] in subtle but
important ways that improve sample quality, and, notably, we directly train our sampler as a latent
variable model rather than adding it after training post-hoc. In greater detail:

1. We use a U-Net with self-attention; NCSN uses a ReﬁneNet with dilated convolutions. We
condition all layers on t by adding in the Transformer sinusoidal position embedding, rather
than only in normalization layers (NCSNv1) or only at the output (v2).

2. Diffusion models scale down the data with each forward process step (by a √1 − βt factor)

so that variance does not grow when adding noise, thus providing consistently scaled inputs
to the neural net reverse process. NCSN omits this scaling factor.
Denoising Diffusion Probabilistic Models