Large Language Models, Quantization, Machine Learning, Data Science, Hands On Tutorials.
use these variables to quantize or dequantize our weights: Let’s take an example: we have a maximum value of 3.2 and a minimum value of -3.0. We can calculate the scale is 255/(3.2 + 3.0) = 41.13 and the zero-point -round(41.13 × -3.0) - 128 = 123 -128 = -5, so our previous weight of 0.1 would be