Generally, BatchNorm sizes shouldn’t be smaller than 32 to get good results. Maybe see the recent GroupNorm paper by Wu & He which references this issue. In the paper itself, I think they got also good results with batchsize 16 in batchnorm, but 32 would be the rule-of-thumb recommended minimum.
https://arxiv.org/abs/1803.08494v1 60