Large Language Models, Quantization, Machine Learning, Data Science, Hands On Tutorials.
vector-wise quantization (row-wise for the hidden state X and column-wise for the weight matrix W). Dequantize the non-outlier results (INT8 to FP16) and add them to the outlier results to get the full result in FP16. Image by author This approach is necessary because 8-bit precision is limited and