I guess you probably didn’t understand what Loss.grad() means here. Because you are doing backprop from Loss all the way backwardly to the leaf nodes in the graph, the starting point is Loss, so Loss.grad() is the grad of loss w.r.t loss itself, so the result will be 1, besides the grad passed to loss which is a scalar defaults to 1 also. Meaning 1 * 1 = 1, so you are getting 1 in the end.
For other nodes, e.g., in the below equation:
y = 2 * x;
if you are calculating x.grad(), which is the grad of loss w.r.t x as in a computation graph, so let’s denote it as dL/dx.
dL/dx = dL/dy * dy/dx = dL/dy * 2;
So if you are calculating Loss.grad(). Which would be:
Loss = Loss. And dL/dL = 1.
So you are getting:
dL/dL = 1 * 1 = 1
As already mentioned by @ptrblck and @gphilip , if you are trying to debug if your model is learning or not, you probably should check the loss value itself.