(Ebtekar, 2021). These evaluations only include users who have tried such competitions, which is a
self-selected subset of all programmers. This is the ﬁrst time that a computer system has achieved
such a competitive level in programming competitions.
We also performed a detailed analysis of our system (Section 6), showing that AlphaCode does not
duplicate sections of code from the training dataset to solve problems, but instead relies heavily on
the natural language problem descriptions to create original solutions. We further examine the types
of problems the model can and cannot solve, and discuss how the validation loss is a poor proxy for
the solve rate.

4

Competition-Level Code Generation with AlphaCode

Figure 3 | Solution to Figure 2 generated by Al-
phaCode. The model successfully extracted the
information necessary to solve the problem from
the natural language description:
alphacode