During training I set  all BatchNorm2d layers to track_running_stats=True, affine=True. I use batch size = 1. During evaluation the batch size is also 1, and what I’ve noticed that if the batch normalization layers are set train=True (i.e. they don’t use means and stds to normalize the image), performance is much better. What am I doing wrong here?