The example input (2, 20) already contains a batch dim, indicating a batch of two 1D examples.

That is not how batchnorm 1d works. Batchnorm 1d assumes an optional first batch dimension, then a channel dimension then an actual dimension. So the input is 2d without batch and 3d with batch.

BTW, I tried to make it of size (1, 2, 20) but it still complains something wrong with the running_mean size:

This is because you define your batchnorm as having 100 channels, but what you give as input has 2.