Hey @Meraki the MASTER_PORT needs to be set to the same value for all processes, otherwise, they cannot conduct rendezvous correctly. This error might be caused by other configurations. It might be helpful to print the value of the following environment variables on all processes right before init_process_group is called: “MASTER_ADDR”, “MASTER_PORT”, “RANK”, “WORLD_SIZE”