Since you are calling the step() method for the corresponding optimizer only in each iteration, you wouldn’t need to detach tensors.



 elkop:

which indicates that gradients aren’t being updated correctly.


You could verify this claim by checking the .grad attribute of the model, which will be updated, and check that it was indeed updated properly in the right iteration.