9

101100101102#params in billions0.00.20.40.60.8Accuracy on LAMBADAOPT Family100101102#params in billionsBLOOM FamilyFP164bit GPTQ4bit RTN3bit GPTQ3bit RTN100101#params in billions910111213141516Perplexity on WikiText2OPT Models 1.3B to 30B4bit4bit/g10244bit/g128FP16Published as a conference paper at ICLR 2023

ACKNOWLEDGMENTS

Elias Frantar and Dan Alistarh gratefully acknowledge funding from the European Research Coun-
cil (ERC) under the European Union’s Horizon 2020 programme (grant agreement No. 805223
ScaleML), as well as experimental support from Eldar Kurtic, and from the IST Austria IT de-
partment, in particular Stefano Elefante, Andrei Hornoiu, and Alois Schloegl. The work of Saleh
Ashkboos and Torsten Hoeﬂer was supported by the PASC DaCeMI project, received EuroHPC-JU
funding under grant MAELSTROM, No. 955513. We thank the Swiss National Supercomputing
Center (CSCS) for supporting us with compute infrastructure.

7 ETHICS STATEMENT
GPTQ