Thank you for your answer.
I found a problem that, compared with the training on more GPUs with the same batch size, this accumulated gradient method cannot solve the batch normalization problem, right?