I use torch.distributed  to train my model.
When I use torch.multiprocessing.set_start_method('spawn'), the gpu usage memory will be increased with the increasing num_workers.
However, when I don’t use  torch.multiprocessing.set_start_method('spawn'), the gpu usage memory is consistent with different num_workers.
Therefore, should I use spawn to start multi-processing ?
What’s the influence of the set_start_method('spawn') ?
Why the increasing num_workers increases the gpu usage memory when spawn mode?