Hi
When testing a model using nn.BatchNorm2d between convolutional layers, I saw that the error (cross entropy loss) over the test set was lower than the error for the training set.
I read some topics on here about testing and I realized I hadn’t used model.eval() before feeding the test set to the model.
What still isn’t clear to me is that in these topics, it is said that BatchNorm2d changes behaviour, when the model is set to model.eval().
My question is the following:
I would think that this behaviour change would imply that for
torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

then “track_running_stats” should be false after setting the model to eval()
However I can see in the summary of model = model.eval() that “track_running_stats” is still true. Shouldn’t this be false, as the batchnorm layer should use the statistics learned from training?
Thanks in advance for any inputs.
Simon