Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
coefficient per-channel for weight W: each column corresponds to a quantization coefficient The combined use of per-token and per-channel can also be called vector-wise quantization. Group-wise M rows (for activations) or K columns (for weights)(e.g. K=64 )correspond to one quantization