mariosasko:

Note: to get a grad of the non-leaf node, use register_hook .


Note that you don’t need to have a full hook. You can just call a.retain_grad() and the .grad field will be populated automatically when you call .backward().It is very similar to the hook, just slightly simpler and you don’t have to define the hook yourself.