you do not have to run rpc on MPI, pytorch distributed provides gloo and nccl backends, you can pass ‘gloo’ or ‘nccl’ to init_process_group().
for rpc, to get better performance, you can use tensor pipe as backend option