Fq

Fq

Algorithm 1 Quantize W given inverse Hessian H−1 = (2XX(cid:62) + λI)−1 and blocksize B.

Q ← 0drow×dcol
E ← 0drow×B
H−1 ← Cholesky(H−1)(cid:62)
for i = 0, B, 2B, . . . do
Q:,j ← quant(W:,j)
E:,j−i ← (W:,j − Q:,j) / [H−1]jj
W:,j:(i+B) ← W:,j:(i+B) − E:,j−i · H

for j = i, . . . , i + B − 1 do

end for
W:,(i+B): ← W:,(i+B): − E · H

−1
i:(i+B),(i+B):

end for

// quantized output
// block quantization errors
// Hessian inverse information

// quantize column
// quantization error
// update weights in block

−1
j,j:(i+B)

// update all remaining weights

5

Published as a conference paper at ICLR 2023

5 EXPERIMENTAL VALIDATION
GPTQ