I have observed that batch normalization parameters such as running mean and running stats get update after we do the forward pass through the model ( as in just after we do output=model(input) ).
So when we train the model and do the evaluation after every epoch is it recommended to put the model in eval mode and then do the inference or not?
Thanks!