Model
Size Solve Rate
Llama 2
7B
Llama 2
13B
Llama 2
34B
Llama 2
70B
Code Llama
7B
Code Llama
13B
Code Llama
34B
Code Llama - Python 7B
Code Llama - Python 13B
Code Llama - Python 34B

14.7%
24.2%
42.2%
56.5%
13.0%
20.8%
32.7%
13.0%
22.1%
34.4%

Table 12: GSM8k results. We report solve rate for Llama 2, Code Llama, and Code Llama - Python
using 7B, 13B, and 34B parameter models. For completeness we also report results with Llama 2 70B
parameters.

in infilling tasks where the prefix does not end in whitespace or a token boundary, or to use the SPM format
in conjunction with token healing.
CodeLlama2