Solved by albanD in post #2 
              
              
                Hi, 
When you do weight = weight - weight.grad*lr, weight now points to a brand new Tensor and so the gradient informations from the original weight Tensor are gone. You can check that after this line, weight.grad is None. 
The other problem you’re going to encounter is that weight = weight - XXX, t…