Thank you for the answer. This should solve the problem. I just need to find a method to iterate all layers within the neural network and add this hook automatically.
I have two follow up questions:

If the activation function is defined in a container like sequential, e.g. nn.ReLU(), does the output consider such activation function or not? In other words, is the output before or after activation function is applied?
Would I be able to combine multiple layers into one? For example, if a layer is followed by batch normalization, can I get the output after batch normalization is applied?