wayi:

Once the two machines forms a process group, you can use collective APIs like allreduce for communication.


I don’t have an idea of the process group and allreduce API. Could you please elaborate on this process or share relevant links to follow up?



 wayi:

You should run these commands on two virtual machines, respectively. There is no “out of two virtual machine”.


Do you mean to run the first distribution command on one VM and the second command on the other VM or run both the commands on both the VMs?