the 10× smaller full-precision 13B model. At 3-bit, RTN collapses completely, while GPTQ can still
maintain reasonable perplexity, in particular for larger models. BLOOM shows a similar pattern: the
gaps between methods are however usually a bit smaller, indicating that this model family might be
easier to quantize. One interesting trend (see also Figure 1) is that larger models generally (with the
exception of OPT-66B2) appear easier to quantize. This is good news for practical applications, as
these are the cases where compression is also the most necessary.
GPTQ