When I train my network on multiple machines (using DistributedDataParallel) I observe my loss exploding when I switch my network to evaluation using model.eval() and torch.no_grad().
When outside the torch.no_grad() context, I switch to model.train() and observe a loss that is way worse than what I was observing at the end of the epoch.
This only happens when using DistributedDataParallel.
Training

the spikes appear at the beginning of the epoch, just after the validation step. The loss at that moment is close to what I observe in validation. Has anyone an idea about what could be causing that ?
Thanks