I just figured out if I used the code below, which do not put layer into .eval() but sets track_running_stats=False only,
I get an error.
for module in model.modules():
    # print(module)
    if isinstance(module, nn.BatchNorm2d):
        if hasattr(module, 'weight'):
            module.weight.requires_grad_(False)
        if hasattr(module, 'bias'):
            module.bias.requires_grad_(False)

        module.track_running_stats = False
        #module.eval()