Solved by ptrblck in post #4 
              
              
                I think your old_dict will just hold a reference to the state_dict and will thus be changed, too. 
Could you add this code snippet to clone the old state_dict and try it again: 
# Save init values
old_state_dict = {}
for key in model.state_dict():
    old_state_dict[key] = self.model.state_dict()[ke…