Remove your last or even 2nd last BN. The BN normalizes feature, the last output is class scores and should not be normalized.
train mode BN uses stat from the batch, test phase it is essentially “cheating” because it accesses to other examples in the batch (hence cannot perform if batch size = 1)