Solved by tom in post #2 
              
              
                Hopefully, you use them in the other order - opt.zero_grad(), loss.backward(), opt.step(). 


zero_grad clears old gradients from the last step (otherwise you’d just accumulate the gradients from all loss.backward() calls).

loss.backward() computes the derivative of the loss w.r.t. the parameters (…