So to be more precise, I would like to know how during training the normalized values are calculated.
Below I have a code snippet which I will be referring to from now on.
If I run the following code, it will always return for the first column:
[-1.2247, 0.0000,  1.2247] which I do not understand. The “right” normalization should be [-1.,0.,1.].
When I use bn.eval(), then the output is correct.
What I would like to know is, what term am I missing during .train() that makes the values so different?
x = torch.tensor([[1.,2.,3.,4.,5.],
                           [2.,2.,2.,2.,2.],
                         [3.,3.,3.,3.,3.]])
bn = nn.BatchNorm1d(5)

[bn(x) for i in range(1000)]