I get the same expected results if both approaches are using eval and training mode, respectively.



 marsggbo:

My question is what is the role of training in F.batch_norm?


During training the batch stats will be used to normalize the input (and the running stats will be updated) while the running stats will be used to normalize the input during eval.
The docs explain this too and also give the formula how the running stats are updated.