Hi,
I did an experiment on how batchnorm works in pytorch. I wrote following code:
import torch
import torch.nn as nn

bn = nn.BatchNorm2d(4)
bn.weight.data = torch.arange(0,4)
bn.bias.data = torch.arange(4,8)
bn.running_mean.data = torch.arange(8,12)
bn.running_var.data = torch.arange(12,16)
inp = torch.autograd.Variable(torch.randn(1,4,4,4))
out = bn(inp)

I set a breakpoint at functional.py#L501-L504 9  and print the running_mean and running_var . I mean as below:
def batch_norm(input, running_mean, running_var, weight=None, bias=None,
               training=False, momentum=0.1, eps=1e-5):
    import pdb
    pdb.set_trace()
    f = torch._C._functions.BatchNorm(running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled)
    return f(input, weight, bias)

I saw a strange malfunction. Always their values were 0 and 1 respectively, regardless of any initialization (as I did above):
(Pdb) running_mean

 0
 0
 0
 0
[torch.FloatTensor of size 4]
(Pdb) running_var

 1
 1
 1
 1
[torch.FloatTensor of size 4]

Could you please check it?