Leaving batch norm training enabled and using batch size 1 makes it behave in a similar way to InstanceNorm w.r.t. training and inference (but you have a different setup what to average over). Depending on the task, that can be a reasonable choice. We usually don’t do that because it is problematic run into problems when inputs are very “dull” (i.e. have very little standard deviaton), they are blown up a lot, so the neural network considered as a function may not be not terribly continuous (well, technically continuity is an “on/off” term, but things can get really “steep”).
Best regards
Thomas