Theoretically, all you need to do quantization is to just round all the weights and set your model to round all the activations and you would obtain a quantized model.
The issue is that we have to choose what numbers to round to. In real life, we round to integers, but that won’t necessarily work well for ML. If we are doing int8 quantization, we only have 256 possible values, forcing us to choose where on the number line to place those rounding points. We want to choose the spacing and range of these 256 possible values to define our rounding process in a way that minimizes the error induced by quantization.
At a high level, the quantization framework ‘observes’ the activations and inputs that come into the model to get a better understanding of the distribution of values and then chooses the spacing and min/max (called scale and zero_point) based on this.
In practice the different observer types take different types of statistics and use that to decide what scale and zero_point to use. This is similar to how a batchnorm operates.