Solved by ptrblck in post #4 
              
              
                This shouldn’t be the case as shown here: 
x = torch.rand(1, 3, 2, 2)
bn = nn.BatchNorm2d(3)
bn.eval()
print(bn.running_mean, bn.running_var)
out = bn(x)
print(out)

bn.running_mean = torch.tensor([100., 100., 100])
print(bn.running_mean, bn.running_var)
out = bn(x)
print(out)