Under the fixed training steps (not epochs), intuitively, train model using batch size=6 on 3GPUs should get a better result. (without batch normalisation)
However, from the results, training using batch size = 4 on a single GPU gets a better result.
That is the part I am confusing.