Hello, hoping for some help as I have been stuck for a long time.  This is a snippet of the code:
optimiser.zero_grad()
loss.backward()
for param in agent.parameters(): param.grad.data.clamp_(-1, 1) 
optimiser.step()
for param in agent.parameters(): param = param.detach()

As you can see, I am already ‘repackaging’ the model parameters.  For context, I am trying to implement DQN, following as closely to the PyTorch tutorial as possible (different environment and use case).
I randomly get one of two errors on each run:
“one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [3, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!”
or
“Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.”
Any suggestions as to what the bug might be?
Also, print(param._version) at the very beginning of the training loop (first line), some of the parameters (weights) are already at version 1, whilst others (biases) are at version 0.  Is that normal?