Does BatchNorm gives good result with very small batch size (different from 1), like batch_size=2

No. BatchNorm needs rather large batches because it essentially assumes that the current batch statistics are a reasonable estimate for the overall dataset statistics.
People have tried to mitigate this by modifying BN training mode into “update the statistics, then use the statistics as in eval mode” but this is quite finicky, in particular at the beginning.
This is why everyone is glad that transformers use layernorm instead. 
Best regards
Thomas