Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
coefficient. Its granularity is between per-tensor and vector-wise. Summary In practical applications, different quantization granularities can be chosen for weights and activations. For example, weights use per-tensor, and activations use per-token. And for activations, there is a distinction