4 THE GPTQ ALGORITHM
Step 1: Arbitrary Order Insight. As explained in the previous section, OBQ quantizes weights in
greedy order, i.e. it always picks the weight which currently incurs the least additional quantization
error. Interestingly, we ﬁnd that, while this quite natural strategy does indeed seem to perform very
well, its improvement over quantizing the weights in arbitrary order is generally small, in particular
on large, heavily-parametrized layers. Most likely, this is because the slightly lower number of
quantized weights with large individual error is balanced out by those weights being quantized
towards the end of the process, when only few other unquantized weights that can be adjusted for
compensation remain. As we will now discuss, this insight that any ﬁxed order may perform well,
especially on large models, has interesting ramiﬁcations.
The original OBQ method quantizes rows of W
independently, in a speciﬁc order deﬁned by the
GPTQ