Solved by mailcorahul in post #2 
              
              
                net.zero_grad() zeroes out grad attribute of every parameter in the net.

loss.backward() computes gradients of every parameter w.r.t loss tensor.

In order to see the gradient of a parameter, you can do print(net.conv1.bias.grad) instead of print(net.conv1.bias.backward). 
backward as you see is …