Could you add logging and make sure that both your processes reach the same state before torch.distributed.init_process_group produces an error?
Also, could you make sure that both processes has w access to the directory that contains rendezvous_file and that the file does not exist before you run torch.distributed.init_process_group
https://pytorch.org/docs/stable/distributed.html#shared-file-system-initialization 11