Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
timing of model quantization can be divided into: Quantization Aware Training(QAT) and Post Training Quantization(PTQ). Quantization Aware Training(QAT) First, we can pretrain the model normally, and insert fake-quantization nodes during fine-tuning(quantize the weights and activation values first,