Figure 2: GPTQ quantization procedure. Blocks
of consecutive columns (bolded) are quantized at
a given step, using the inverse Hessian informa-
tion stored in the Cholesky decomposition, and
the remaining weights (blue) are updated at the
end of the step. The quantization procedure is
applied recursively inside each block: the white
middle column is currently being quantized.

col, d3

4

Inverse Layer Hessian(Cholesky Form)computed initiallyblock i quantized recursivelycolumn-by-columnWeight Matrix / Blockunquantized weightsthat are updatedquantized weightsPublished as a conference paper at ICLR 2023
GPTQ