arshagarwal:

Also, how to ensure before running the training script on each remote node that the 2 remote nodes have access to each other?


Can you try ssh to one of the remote machine and then ping/ssh another remote machine?

Specifically, given a local machine with public IP address “Ip0” and 2 remote nodes with Public IP address “Ip1” and “Ip2” respectively on which training is to be performed remotely from the local machine, how to go about making such a set-up?

One of the remote machine can serve as the master, i.e., using it’s IP address as the MASTER_ADDR and pick a port for MASTER_PORT.