E Zero shot results on APPS

In addition to two-shot results we report in Table 3, we also list the zero-shot performance for Code Llama -
Instruct in Table 15. For both the two-shot and zero-shot results, we use nucleus sampling (p = 0.95) at
temperature 0.6 for all of our models. The prompt templates are shown in 13. We prompt the model to wrap
the final code answer inside of triple single quotes, which makes it easier to extract the answer. We use a
special instruction to help models understand the specific question format: “read from and write to standard
IO” for standard questions and “use the provided function signature” for call-based questions, which we insert
into our prompt as the question guidance. Despite not finetuned on the training data nor provided with few
shot examples, Code Llama - Instruct can achieve convincing results on these challenging competitive
programming questions.

29

Size

Introductory

Interview

Competition
CodeLlama2