Hi,
Are you sure that you don’t keep some states from one iteration to the next?
Also when you do loss = loss.detach().requires_grad_(True) you break the link between the loss value and the rest of the net. So doing loss.backward() is not going to do anything.