Solved by mrshenli in post #2 
              
              
                Hey @rahul003, 
Distributed autograd is using RPC, so you need to call init_rpc instead of init_process_group. See the toy example below: 
import torch.multiprocessing as mp
import torch.distributed.rpc as rpc
import torch.distributed.autograd as dist_autograd
import os
import torch
from torch impor…