Yes that is true.
What I meant was using autograd function, on a given tensor and get it’s gradient update value.
Generally what we do is, we have this leaf node or loss value and call backward().
So, it traverse back and calculate all gradients for us. Is this right?
If so, I can access w.grad and get the weight updates.
What if I manually go back and just go layer by layer by calling corresponding grad_function of each tensor (contributing to weights) by passing shape-matching inputs and get the grad value.
This way, I get more control over it. I don’t want to entirely loose the autograd functionality, but use it at node level to just get the corresponding weight vector update. Can this be possible?