gphilip:

since (as far as I understand) calling loss.backward() has no effect on loss.grad.


For the sake of completeness:
The backward() operation will use the default gradient value of 1 for scalar loss values (for other shapes, you need to manually pass the gradient to backward).
Here is a small example showing that loss.grad is defined by the passed gradient:
model = nn.Linear(1, 1)
x = torch.randn(1, 1)

# default gradient
out = model(x)
loss = out**2
loss.retain_grad()

loss.backward()
print(loss.grad)
> tensor([[1.]])

# pass gradient manually
out = model(x)
loss = out**2
loss.retain_grad()

loss.backward(torch.tensor(10.).view_as(loss))
print(loss.grad)
> tensor([[10.]])

Besides that, I agree with @gphilip that checking the loss.grad value is not particularly useful to check if the model is learning or not.
As a debugging step I would also suggest to check the .grad attribute of all parameters of the model and make sure they show valid (or the expected) values.