ptrblck:

retain_graph=True


So, the retain retain_graph = True is necessary for the first call to backward(), i.e. actor_loss.backward(), and the error is still present if I remove it in the the second call to backward().
The error only occurs when I implement the LSTM functionality. I’m not sure if it works this way, but could it have something to do with the fact that state tensors are used more than once because they are part of a sequence and, therefore, the gradients are accumulating in individual state leaf tensors? However, I still don’t know if that makes sense because it doesn’t seem to have anything to do with inplace operations.
Thanks for your help.