27.6
28.0
21.7
18.5
18.9
20.1
18.9
23.6
23.2
21.7
19.3
23.2

23.6
27.7
26.4
32.7
22.3
29.5
35.0
41.8
28.2
27.3
32.7
41.8

18.7
23.9
16.1
19.6
26.1
22.2
18.7
23.9
23.9
23.0
21.3
25.7

23.0
28.7
22.3
36.4
24.6
34.9
37.3
42.1
31.0
38.1
40.7
48.8

21.2
35.1
17.3
40.2
19.2
31.6
48.0
56.7
22.4
41.0
47.5
70.2

20.8
37.9
20.4
45.7
21.9
39.8
59.5
63.6
32.7
54.6
62.1
76.6

25.2
63.1
26.2
58.7
33.0
52.9
74.8
83.0
43.2
62.1
77.2
86.9

50

Model
MPT

Falcon

Llama 1

Llama 2

Size GSM8k MATH
7B 6.8
30B 15.2
7B 6.8
40B 19.6
7B 11.0
13B 17.8
33B 35.6
65B 50.9
7B 14.6
13B 28.7
34B 42.2
70B 56.8

3.0
3.1
2.3
5.5
2.9
3.9
7.1
10.6
2.5
3.9
6.24
13.5

Table 25: Comparison to other open-source models on mathematical reasoning tasks, GSM8k and MATH
(maj1@1 is reported).

Mathematical Reasoning.
GSM8k and MATH tasks.

In Table 25, we report results for Llama 2 and other open-source datasets on the
Llama2