Hi,
I’m so sorry that I didn’t explain it well.
Also, I changed my code so that you can execute it without any additional files.
So what I’m doing in the code is that

I put a randomized image into resnet50
(layers of resnet50 is as following)

conv1, bn1, relu, maxpool, layer1, layer2, layer3, layer4, avgpool, fc

I picked the output of layer3 and get the gradient map of it. (gradient of a single score with respect to the output of layer3)
and I summed all the elements of the gradient map. Let’s call this as grad_sum_loss
And I get the gradient of grad_sum_loss with respect to “layer4[0].conv1.weight”. Note that it’s layer4 not layer3.

So when you see the output, there are two register_hook results.
Both of them are from the register_hook of the weight of “layer4[0].conv1.weight” but each of them are from different .backward().
The register_hook first called when the gradient of a single score with respect to the output of layer3 is calculated.
The register_hook secondly called when the gradient of grad_sum_loss with respect to “layer4[0].conv1.weight” is calculated.
I used for loop to calculate numerical gradient of each weights of “layer4[0].conv1.weight”.
You can see nemerical_target_feature_map_grad_sum_grad in the output which is the numerical gradient of grad_sum_loss with respect to “layer4[0].conv1.weight”.
The problem is that this is different from the second register_hook result.
(I also calculated the numerical gradient of a single score with respect to “layer4[0].conv1.weight” but this is identical to the first register_hook result)