Solved by zhl515 in post #4 
              
              
                Yes, you can get the gradient for each weight in the model w.r.t that weight. Just like this: 
print(net.conv11.weight.grad) 
print(net.conv21.bias.grad)

The reason you do loss.grad it gives you None is that “loss” is not in optimizer, however, the “net.parameters()” in optimizer. 
optimizer = opti…