and maximum submission time penalties. The bounds represent how ranking depends on the number
of accelerators used to draw samples during competition. For the second and third runs, Table A6
shows the estimated percentile when not limiting to 10 submissions per problem (still taking into
account penalties for incorrect submission), which although not human-like does follow competition
rules. We found that the model still continued to solve problems when given more attempts, though
at a decreased rate. The model tends to solve the easier problems in competitions, but it does manage
to solve harder problems including one rated 1800.
Overall our system achieved an average ranking of top 54.3% limiting to 10 submissions per problem,
with an actual average of 2.4 submissions for each problem solved.9 When allowed more than 10
submissions per problem (the second and third evaluation), AlphaCode achieved a ranking of top
alphacode