Large Language Models, Quantization, Machine Learning, Data Science, Hands On Tutorials.
plot the distribution of the dequantized and original weights. If the quantization is lossy, it would drastically change the weight distribution. The following figure shows this comparison, where the blue histogram represents the original (FP32) weights, and the red one represents the dequantized