Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
understand the meaning of different quantization granularities by multiplying the activation matrix X and the weight matrix W. Per-tensor This is the simplest way, and also the largest granularity, the entire activation matrix corresponds to a quantization coefficient s, the same is true for the