S8. PickScore Rejection Sampling
Rejection sampling was used in [21] as a powerful inference-time tool. 100 samples were drawn from variants of a prompt
and PickScore-ranked, with the highest scored images being compared to a single random draw. PickScore selections were
human-preferred 71.4% of the time. We compare using additional compute at inference vs. additional training in Figure S4.
We plot the expected PickScore win rate of n draws from the reference model against a single draw from the learned (DPO)
model. The mean inference compute for baseline rejection sampling to surpass the DPO-trained model is 10× higher in both
cases. For 7% (SDXL) and 16% (SD1.5) of the prompts even 100 draws is insufficient.

Figure S3. Box plots of automated metrics vs. SDXL baseline. All 500 unique prompts from PickScore validation set.
DiffusionModelAlignmentUsing Direct Preference Optimization