Hi,
I am doing an experiment where I use the output of intermediate values of batchnorm layer from resnet50. So, I forward hook at different layers and work on the output. But, the weird thing I observed is the min value of the batchnorm output clamps to zero at all the layers. To reproduce the result I wrote a toy program:
import torch
from torchvision import models, datasets
import torchvision
import datasets
from tqdm import tqdm, trange


activation_student = {}
def get_activation_student(name):
    def hook(self, input_, output):
        activation_student[name]=  output.data
    return hook

amazonData = datasets.amazon_load('train', 16)
model = models.resnet50(pretrained=True)
for n, m in model.named_modules():
    if n == 'layer2.1.bn1' or 'layer3.2.bn2' or 'layer4.1.bn3':
        m.register_forward_hook(get_activation_student(n))

for i in trange(5, leave=False):
    source_x, source_y = next(iter(amazonData))
    out = model(source_x)
    print("min: ", torch.min(activation_student['layer2.1.bn1']))
    print("min: ", torch.min(activation_student['layer3.2.bn2']))
    print("min: ", torch.min(activation_student['layer4.1.bn3']))


output:
min: tensor(0.)
min: tensor(0.)
min: tensor(0.)
can someone help me in fixing this, or is this expected ?