Arun_Vishwanathan:

Basically how to ensure that in eval mode, it uses only previous stats and does not use anything from the current batch?


Leave track_running_stats=True and set the batchnorm layer to eval().
This setup will update the running stats in train() mode and just use them (without updating) in eval() mode.
If you set track_running_stats=False, the batch statistics will always be used as explained in the docs.