Hi!
I have something like the following in a loss function (this is code from an exercise, so there’s no direct practical value):
old_state = copy.deepcopy(model.state_dict())
logits = model(x)
loss = objective(logits, y)
model.load_state_dict(old_state)
return loss

This leads to an error in the backward step of the loss:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [300, 10]], which is output 0 of TBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

The anomaly detection isn’t helpful in this case. Commenting out the model.load_state_dict statement removes the error message.
Any ideas on how I could save and load the state_dict without running into an error?