Solved by SimonW in post #2 
              
              
                There are the gamma and betas. They are initialized as gamma ~ U[0, 1], and beta <- 0. if you do 
x = torch.FloatTensor(1,1,2,2)
x[0,0,0,:] = 1
x[0,0,1,:] = 2
m = nn.BatchNorm2d(1)
m.weight.data.fill_(1)
m.bias.data.zero_()
y = m(Variable(x))
# you will get
>>> y

(0 ,0 ,.,.) =
 -1.0000 -1.0000
  1.…