I don’t think it’s a bug in pytorch, as I’ve commented in the other issue.
Using batch norm with such small batch sizes is probably not a good idea. If you need those (because of using a pre-trained network), I’d freeze the mean/std and parameters of the batch norm