4.4. Large scale sampling
Sampling from transformer models can be easily parallelized, which allowed us to scale to millions
of samples per problem – a critical driving force for performance improvement. To ensure suﬃcient
diversity in such a large number of samples, we take a single trained model and: (i) generate half of
the samples in Python and half in C++, (ii) randomize the problem tags and ratings in the natural
language prompt (see Figure 5 for an example and Appendix C.2 for more details), and (iii) use a
relatively high sampling temperature. The single model, via the additional metadata we condition
upon, can generate solutions with diﬀerent languages, tags, and ratings. To make the most eﬀective
use of our samples we then apply ﬁltering (Section 4.5) and clustering (Section 4.6) to obtain a small
number of candidate submissions.
For problem tags and ratings conditioning, we picked random tags from the most popular 50 for the
alphacode