If your model is symbolically traceable we would recommend using (prototype) FX Graph Mode Post Training Static Quantization — PyTorch Tutorials 1.10.0+cu102 documentation 2
performance might be related to the hardware you are running on as well, we may be able to help if you can provide more details on where do you run the model, is it on server or mobile, what OS/CPU architecture does it use?