Hi mohit7,
Make sure you create the net using previous definition, and let the net go through process that was applied during quantization before (prepare_model, fuse_model, and convert), without rerun the calibration process.
After that you can load the quantized state_dict in. Hope it helps.