(d) Top-k sampling: pass@k

(e) Nucleus sampling: 10@k

(f) Nucleus sampling: pass@k.

Appendix Figure A5 | Sampling with temperature, top-k sampling and nucleus sampling. (a-b)
Lower temperatures are better for small sample budgets, and higher temperatures are better for large
sample budgets. However, the model is relatively tolerant to a wide range of temperatures. (c-d)
Top-k sampling does not outperform the sampling with temperature baseline. (e-f) Nucleus sampling
does not signiﬁcantly improve the sampling baseline. Performance increases with the nucleus size,
with an optimal size close to 1.0 (i.e. plain sampling).

48
alphacode