Solved by ptrblck in post #2 
              
              
                In training mode nn.BatchNorm layers will use the batch statistics and update the running estimates. 
If you call .eval() on this layer or your complete model, the running estimates will be used. 
Are you comparing a training run of your model with a manual eval implementation?