Hey thanks for the reply. I’ve thought about that too but if it was done for the purpose of reducing computation, shouldn’t it be

backward hook in module layer4/2/relu
backward hook in module layer4/2/bn3
backward hook in module layer4/2/conv3

which is the expected order as gradient props backward.
I’ve tried the register hook code on another toy model,
class test_model(nn.Module):
    def __init__(self):
        super(test_model, self).__init__()
        self.conv1 = nn.Conv2d(3, 8, [3, 3])
        self.bn1 = nn.BatchNorm2d(8)
        self.conv2 = nn.Conv2d(8, 3, [3, 3]) 
        self.bn2 = nn.BatchNorm2d(3) 
        self.relu = nn.ReLU()
        self.avgpool = nn.AdaptiveAvgPool2d(3)
        self.linear = nn.Linear(3, 1)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)
        out = self.relu(out)
        out = self.avgpool(out)
        out = self.linear(out)
        return out

for this model the result is as expected

forward hook in module conv1
forward hook in module bn1
forward hook in module relu
forward hook in module conv2
forward hook in module bn2
forward hook in module relu
forward hook in module avgpool
forward hook in module linear
backward hook in module linear
backward hook in module avgpool
backward hook in module relu
backward hook in module bn2
backward hook in module conv2
backward hook in module relu
backward hook in module bn1
backward hook in module conv1

so I really have no idea what’s going on there…