Solved by jpeg729 in post #2 
              
              
                The leaf nodes are the weight and bias parameters of the Linear module. 
Whenever you run 
out = net(input)
loss = loss_fn(out, target)
loss.backward()

The gradients of the loss with respect to the weight parameter of the Linear module are added to net.weight.grad. 
Note that running loss.backward(…