I seem to have the same issue in my binary classification. When evaluating in .train() mode my model gets a high accuracy on my test data. Whereas, when I set the model to .eval() mode to switch dropout mode etc. my model always outputs class 0.
Switching everything but the BatchNorm layers to eval() mode gives me a good prediction. Unlike @bartholssthoorn my mini-batches contain already both classes (i.e. shuffled) so that would not solve it.
My running_means do not contain any NaNs.
What else can I check to figure out why the batchnorm layers are preventing my evaluation mode?