I have a network that consists of batch normalization (BN) layers and other layers (convolution, FC, dropout, etc) which is pretrained ResNet50 model.
I want the model not to be trained so I freezed the all layer with requires_grad=False,
but I find the BN layer still updating and the performance gradually dropped.
So, I used the below code to freeze the batch norm layer.
for module in model.modules():
    # print(module)
    
   if isinstance(module, nn.BatchNorm2d):
 
        if hasattr(module, 'weight'):
            module.weight.requires_grad_(False)
        if hasattr(module, 'bias'):
            module.bias.requires_grad_(False)
        module.track_running_stats = False
        # module.eval()



But I am confused about the difference between putting layer.eval() and module.track_running_stats=False.


If I want the BN layer not to be updated and performance not to be changed in inference, do I just need to make it module.weight.requires_grad(False) only? Or should I also stop track_running_stats
which is used in inference mode.


Thank you.