isalirezag:

when i put loss.register_hook(lambda grad: print(grad)) before loss.backward() it gives me tensor(1., device='cuda:0') , is it what it is supposed to show? what intermediate values it is computing the gradient wrt?


Yes, as you are calculating dLoss/dLoss = 1. Note that you can also pass this gradient directly to your backward call.