hello,


how exactly you performed QAT? , did you fine tune the model or trained it from scratch? fine tune is better , are you quantizing all layers in the model? , there might be operations in the first layers that once quantized make the model generalization performance decrease.


I think once you have done QAT, you don’t need a representative dataset to calibrate the range of the activations, just run the converter with converter.optimizations = [tf.lite.Optimize.DEFAULT] like it is done here in the docs: Quantization aware training comprehensive guide 5


you can check the model layers details in the Netron app, I use it to debug and see if the layer inputs/outputs and weights make sense after quantization.