One example could be the availability of hook registration as below.

  
    
    
    How to register hook function for functional form? 
  
  
    I’m now trying to implement Guided backpropagation from https://arxiv.org/abs/1412.6806 , which alters the gradient calculation in ReLU layer. 
When given model is implemented in the way containing nn.ReLU() instance as its submodule, I can easily register backward hook function to its every submodule as below. 
model = resnet101(pretrained=True)
def hook(module, grad_input, grad_output):
    if isinstance(module, nn.ReLU):
        # do something
        return changed_grad_input
for i, v in mod…