Solved by mrshenli in post #2 
              
              
                Hey @Amit_Singh1, can you try if spawn mode works for you? I tried the following, and it works in my dev env. 
import os
import torch.distributed as dist
import torch.multiprocessing as mp

def run(size, rank):
    print(size, rank)

def init_process(rank, size, fn, backend="gloo"):
    """Initializ…