No, it is overlapped with gradient computation (backward propagation) only. As more and more gradients have been computed, they are ready to be reduced. There is no need to wait with reducing them until you have computed all of them.