The .train() and .eval() call on batchnorm layers does not freeze the affine parameters, so that the gamma (weight) and beta (bias) parameters can still be trained.



 Rakshit_Kothari:

I understand that the eval operation allows us to use the current batch’s mean and variance when fine tuning a pretrained model.


Calling eval() on batchnorm layers will use the running stats, while train() will use the batch stats and update the running stats.