Solved by pietern in post #4 
              
              
                If you use vanilla multiprocessing you’ll have to do this yourself. If you use it in combination with torch.nn.parallel.DistributedDataParallel then gradient synchronization and averaging is done for you. Also see the documentation on torch.distributed for more information.