Thanks Can!

You are welcome!

OK. looks like it works now. To run on multiple machines, i assume we dont’ need to start an etcd instance first. Just simply set rdzv_endpoint=“IP_ADDRESS” rdzv_backend=“c10d”. And it works. Looks like c10d backend takes care of everything for us.

That is right. The new c10d rendezvous backend does not depend on any 3rd party software. You don’t need to start (or even install) etcd.
From the output it looks like your machines run in silo. Each has a world size of 1 and rank 0.
Since you are using our new launcher API you should not set any environment variables in your training script. Our launcher sets all four variables you listed before executing it.
I suggest checking out our “Distributed Elastic” docs for v1.9 RC here 1 which also lists all the environment variables set for you by the launcher API.
Let me know if you still have trouble after reading the docs.
Cheers.