From what I understand, loss.retain_grad() initializes loss.grad to 1.  This is why you get 1 as the value of loss.grad. This understanding was wrong. See my comment below.
This has no relation to whether your network is learning or not, since (as far as I understand) calling loss.backward() has no effect on loss.grad.
A better way to figure out if your network is learning is to print out one of the values in loss and see how if it changes over iterations.