Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
activation values (i.e., the inputs of each layer), they are unknown in advance, depending on the specific inference input, which is more tricky. According to the quantization of activations, PTQ can be divided into dynamic and static quantization. Dynamic Quantization: During the inference