/usr/local/lib/python3.7/dist-packages/torch/quantization/observer.py:990: UserWarning: must run observer before calling calculate_qparams.                                    Returning default scale and zero point
Returning default scale and zero point "

This error message means that there are observers in the network which have not been calibrated.  If observers are not calibrated, then they will use scale=1.0 and zero_point=0, which is probably not going to be useful.  Could you verify that your calibration is working correctly, and all observers have scale and zero_point collected based on your calibration data?