Hey @albanD, thanks for taking the time to answer. I have a few follow up questions.


What do you mean when you say intermediate results? Gradients are calculated for a weight  (dL/dW). By intermediate results do you mean dL/dA where A is the activation for applied for a layer which has a weight W? Can show what these intermediate results are in the following image please?


How is register_hook applied on a tensor different from register_backward_hook() applied on a nn.Module? What is nn.Module here - a layer such as nn.Conv2d or an entire class like CNNModel()? I have this confusion because one example in the doc has this net.conv2.register_backward_hook(printnorm) (used on a layer conv2) but modules are supposed to be classes that have a .forward() function. Or is my understanding of a nn.Module incorrect?


Can you highlight where register_hook(), register_forward_hook(), register_backward_hook() apply in the attached image please?
Sample Neural Net1170×426 56 KB
In the code example you wrote, register_hook() is applied after self.cnn1. So, the hook (it’s like a callback function right?) can now access the gradients flowing into self.cnn1 during backprop?