blueskywwc:

Why the accuracy becomes higher after quantization?


We don’t expect accuracy to increase due to quantization, this is likely random variation. To test this theory, you could run evaluation on various slices of data unseen in training. I would expect the mean difference of accuracy on a set of slices would be a slight drop for the quantized model compared to the floating point model.