40.36
38.55
37.92
36.07
36.91

2.7B
54.34
52.90
53.11
26.05
48.19

6.7B
60.14
57.68
59.72
25.04
53.41

13B
61.83
61.31
61.32
30.60
56.82

30B
65.40
61.11
65.11
34.22
59.72

66B
67.26
40.66
65.35
25.84
52.44

175B
71.04
63.93
68.69
26.77
65.36

Table 17: OPT accuracy on ARC-easy.

15

Published as a conference paper at ICLR 2023

BLOOM Bits
16
full
RTN
4
4
GPTQ
3
RTN
GPTQ
3

560M 1.1B
45.41
41.71
39.40
42.51
44.49
40.24
46.87
45.44
39.14
41.79

1.7B
48.11
44.70
44.49
37.58
42.85

3B
53.24
51.35
52.82
45.08
46.63

7.1B
57.37
56.14
56.14
48.61
51.56

176B
67.47
66.33
67.42
28.87
62.84

Table 18: BLOOM accuracy on ARC-easy.

OPT
full
RTN
GPTQ
RTN
GPTQ

Bits
16
4
4
3
3

125M 350M 1.3B
29.44
22.87
24.91
22.44
22.95
28.24
23.55
21.76
22.53
27.65

24.06
23.81
24.83
22.18
25.09

2.7B
31.31
29.18
30.12
25.43
27.82

6.7B
34.56
32.59
33.70
25.85
31.91

13B
35.75
35.24
34.90
23.81
33.02

30B
38.14
35.41
37.80
19.97
35.84

66B
40.02
22.87
39.16
25.77
31.66

175B
43.94
37.71
42.75
23.81
41.04
GPTQ