I trained a model with pytorch 1.2 on a distributed GPU system. And the training step on each code is out-sync. For example,  the node 1 of the distributed system has already finished the training task on the epcho N, while node 2 just start the training step N-1. I wonder wether that the out-sync phenomenon would make negative effectiveness on model training. I will appreciate if anyone could give me some suggestion.