Quantization, Large Language Models, Llm, Deep Learning, ChatGPT.
Unlike QAT, which requires sufficient training data for retraining, PTQ can be applied in situations where data is limited or unlabeled. However, compared to QAT, especially for low-precision quantization, PTQ often comes at the cost of lower accuracy. Granularity of Model Quantization Quantization