Solved by albanD in post #6 
              
              
                Yes. 
So maybe it’s simpler to consider the autograd.grad(loss, w) API first: 
Given the output and input of a function, it will return the gradient dloss/dw to you. 
In the context of torch.nn, to avoid the user the burden of moving these gradients around, we use loss.backward() that will populate …