Hello all,
I ran into a similar problem - I am using BatchNorm1d with a batch size of 1, which always results in running_vars which are NaN’s. Specifically, this only occurs with a batch of size 1.
This problem doesn’t occur with BatchNorm2d.
I thought it was possibly due to the eps value as someone suggested above, but this wouldn’t explain why it’s ok for 2d cases and why it doesn’t produce NaN’s for the first stddev calculation.
EDIT:

I presume the NaN isn’t a result of performing 1 / (0 + eps)? Where the 0 arises because it is computing the variance from a single example.

For example:
input = torch.FloatTensor(1,4).normal_(0,1)
bn = nn.BatchNorm1d(4)

output = bn(Variable(input))

print("output ...\n", output)
print("running mean ...\n",bn.running_mean)
print("running var ...\n",bn.running_var)

produces:

output …
Variable containing:
0  0  0  0
[torch.FloatTensor of size 1x4]


running mean …
0.0437
0.0830
0.0557
0.1216
[torch.FloatTensor of size 4]


running var …
nan
nan
nan
nan
[torch.FloatTensor of size 4]

Have I missed something obvious?
Cheers,
Jordan