Solved by albanD in post #2 
              
              
                Hi, 
Unfortunately I’m not sure if it is easy to do this. 
I guess you could always replace F.relu by a custom function that would use the nn.Module version with your hook. 
Otherwise, you can add a hook on any tensor you want, but that would require modifying the forward method of your module.