I think the fix here is to ensure you always have a batch dimension. Potentially adding an  .unsqueeze(0)  to your input.

The example input (2, 20) already contains a batch dim, indicating a batch of two 1D examples.
If we fake the input with unsqueeze(0), how could it work when there are other modules before BatchNorm1d in the model that may assume the 0th dim must be the batch dim?
After all, the layer(s) before BatchNorm1d can be anything else in general, right?
BTW, I tried to make it of size (1, 2, 20) but it still complains something wrong with the running_mean size:
Traceback (most recent call last):
  File "syncBN.py", line 29, in <module>
    output = module(input)
  File "/home/ml/farleylai/miniconda3/envs/sinet37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ml/farleylai/miniconda3/envs/sinet37/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/ml/farleylai/miniconda3/envs/sinet37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ml/farleylai/miniconda3/envs/sinet37/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 459, in forward
    exponential_average_factor, self.eps)
  File "/home/ml/farleylai/miniconda3/envs/sinet37/lib/python3.7/site-packages/torch/nn/functional.py", line 1670, in batch_norm
    training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: running_mean should contain 2 elements not 100


This would be a nice addition, we would be happy to merge a PR that adds this feature!

So you are suggesting it is not intentional but something that could be completed?
If that is the case, it seems like creating an issue on the GitHub repo makes more sense and I will look into the details under the hood.
Therefore to sum up, there is likely no luck for those with BatchNorm1d already to be converted to SyncBN transparently by default.