Hello,
I am trying to get started with torch.distributed with the following toy example, on a multi-gpu cluster :

  
      github.com
  
  
    narumiruna/pytorch-distributed-example/blob/master/toy/main.py 12
import argparse
from random import randint
from time import sleep

import torch
import torch.distributed as dist


def run(world_size, rank, steps):
    for step in range(1, steps + 1):
        # get random int
        value = randint(0, 10)

        # group all ranks
        ranks = list(range(world_size))
        group = dist.new_group(ranks=ranks)

        # compute reduced sum
        tensor = torch.tensor(value, dtype=torch.int)
        dist.all_reduce(tensor, op=dist.ReduceOp.SUM, group=group)


  This file has been truncated. show original

  
  
    
    
  
  


After running the program with the following command :
python3 main.py --init-method tcp://127.0.0.1:23456 --rank 0 --world-size 2

The program gets stuck in an the  dist.init_process_group on line 42. I am not really sure about the reason as no message gets displayed.
Thanks,