3

Published as a conference paper at ICLR 2023

whose Hessian is HF = 2XF X(cid:62)
F , where F denotes the set of remaining full-precision weights,
the greedy-optimal weight to quantize next, which we denote by wq, and the corresponding optimal
update of all weights in F , denoted by δF , are given by the following formulas, where quant(w)
rounds w to the nearest value on the quantization grid:

wq = argminwq

(quant(wq) − wq)2

[H−1

F ]qq

,

δF = − wq − quant(wq)

[H−1

F ]qq

· (H−1

F ):,q.

(2)

OBQ quantizes weights iteratively using these two equations, until all the weights of w are quan-
tized. This is done efﬁciently, avoiding expensive full recomputations of H−1, by removing the qth
row and column of H, which is necessary after quantizing wq, directly in the inverse via one step of
Gaussian elimination. Namely, the updated inverse is given by the formula

(cid:16)

(cid:17)

H−1−q =

H−1 −

1

[H−1]qq

H−1
:,q H−1

q,:

.

−p

(3)
GPTQ