Thanks for the explanation.
Actually I trained the model again from scratch and saved the state_dict like torch.save(model.state_dict(), fn) and load the state_dict accordingly. However I am getting this error when I try to test the model:
Traceback (most recent call last):
  File "pytorch_src/test_evaluate.py", line 144, in <module>
    model_load(args.save)
  File "pytorch_src/test_evaluate.py", line 109, in model_load
    model.load_state_dict(torch.load(fn))
  File "/home/anaconda3/envs/py36/lib/python3.7/site-packages/torch/nn/modules/module.py", line 839, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for RNNModel:
        Unexpected key(s) in state_dict: "rnns.0.module.weight_hh_l0", "rnns.1.module.weight_hh_l0", "rnns.2.module.weight_hh_l0".

I guess there must be a mismatch between model and the state_dict keys but not sure how that happened.
‘rnns.0.module.’ prefix get added to original keys I guess. Do you have any idea?