mrshenli:

it needs to quantitatively measured, as it is affected by a lot of factors, e.g. batch size, lr, loss function, etc.


Ok I see. So It can be really dangerous to say that DDP is better or DP is better. I will just keep it and do not put into my the paper. Anyway I will cite your paper since I am using DDP.