2021) performs direct optimization based on straight-through estimates. BRECQ (Li et al., 2021)
introduces Fisher information into the objective, and optimizes layers within a single residual block
jointly. Finally, Optimal Brain Quantization (OBQ) (Frantar et al., 2022) generalizes the classic
Optimal Brain Surgeon (OBS) second-order weight pruning framework (Hassibi et al., 1993; Singh
& Alistarh, 2020; Frantar et al., 2021) to apply to quantization. OBQ quantizes weights one-by-one,
in order of quantization error, always adjusting the remaining weights. While these approaches can
produce good results for models up to ≈ 100 million parameters in a few GPU hours, scaling them
to networks orders of magnitude larger is challenging.
Large-model Quantization. With the recent open-source releases of language models like
BLOOM (Laurenc¸on et al., 2022) or OPT-175B (Zhang et al., 2022), researchers have started to
GPTQ