Hi all ,
I try to run pytorch with distributed system.
I run test1.py as below

import torch
import torch.distributed as dist
def main(rank, world):
if rank == 0:
x = torch.tensor([1., -1.]) # Tensor of interest
dist.send(x, dst=1)
print(‘Rank-0 has sent the following tensor to Rank-1’)
print(x)
else:
z = torch.tensor([0., 0.]) # A holder for recieving the tensor
dist.recv(z, src=0)
print(‘Rank-1 has recieved the following tensor from Rank-0’)
print(z)
if name == ‘main’:
dist.init_process_group(backend=‘mpi’)
main(dist.get_rank(), dist.get_world_size())

Then I run with single machine.

mpiexec -n 2 python test1.py

Finally, the error is
Traceback (most recent call last):
  File "test1.py", line 17, in <module>
    dist.init_process_group(backend='mpi')
  File "/cluster/home/cnphuong/my_environment/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 392, in init_process_group
    timeout=timeout)
  File "/cluster/home/cnphuong/my_environment/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 452, in _new_process_group_helper
    raise RuntimeError("Distributed package doesn't have MPI built in")
RuntimeError: Distributed package doesn't have MPI built in
Traceback (most recent call last):
  File "test1.py", line 17, in <module>
    dist.init_process_group(backend='mpi')
  File "/cluster/home/cnphuong/my_environment/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 392, in init_process_group
    timeout=timeout)
  File "/cluster/home/cnphuong/my_environment/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 452, in _new_process_group_helper
    raise RuntimeError("Distributed package doesn't have MPI built in")
RuntimeError: Distributed package doesn't have MPI built in
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[31741,1],1]
  Exit code:    1
--------------------------------------------------------------------------

I also installed pytorch with

pip install torch torchvision

Please help me.
Thanks,