Solved by supriyar in post #2 
              
              
                While loading the model is the model now a quantized model? If you convert the model to quantized model and then load the quantized state_dict it should work.