Just found out that this is because it doesn’t save the dictionary used by [optimizer] (https://github.com/pytorch/pytorch/blob/master/torch/optim/optimizer.py 84) as it is while using optimizer.state_dict(). Before saving the optimizer, it’s state variable is defaultdict(<type 'dict'>, {}) but when it is loaded back it changes to a simple dict {}