models. Adaptive rounding-based methods typically employ a lot more SGD steps and would thus
be even more expensive (Nagel et al., 2020; Li et al., 2021).
Language Generation. We begin our large-scale study by compressing the entire OPT and BLOOM
model families to 3- and 4-bit. We then evaluate those models on several language tasks including
WikiText2 (Merity et al., 2016) (see Figure 1 as well as Tables 3 and 4), Penn Treebank (PTB) (Mar-
cus et al., 1994) and C4 (Raffel et al., 2020) (both in Appendix A.3). We focus on these perplexity-
based tasks, as they are known to be particularly sensitive to model quantization (Yao et al., 2022).
On OPT models, GPTQ clearly outperforms RTN, by signiﬁcant margins. For example, GPTQ loses
only 0.03 perplexity at 4-bit on the 175B model, while RTN drops 2.2 points, performing worse than
the 10× smaller full-precision 13B model. At 3-bit, RTN collapses completely, while GPTQ can still
GPTQ