I’ve trained the stock torchvision ResNet50 to predict classes for images. It achieves 93% accuracy in training.
When I load my model after training and place it in eval mode it gives completely different results, and so accuracy for the same images. The model in .train() mode gives the expected ~93% but in .eval() mode it gives only 50%, as if it hasn’t been trained at all.
I can force the two models (in train and in eval modes) to behave similarly if I run:
for module in model.modules():
    if isinstance(module, torch.nn.modules.BatchNorm2d):
        module.running_mean = None
        module.running_var = None

where model is the model in eval mode.
I now get the correct outputs from the .eval() model but only if I use the same batch size as in training (64). The output should not depend on batch size.
Is this an issue with the BatchNorm implementation, or am I doing something wrong?