The nodes become aware of each other through a process called rendezvous, which happens within dist.init_process_group which is a synchronization point for all nodes. Looking at the arguments you’ve passed into init_process_group, if your client is on a different machine and your filesystem is not somehow networked, different files will be used for initialization, so the processes will never come to know about each other.
If you are using windows for DDP training, we only support file-backed initiliazation and single-machine use cases. We have landed TCP-based initialization support in PyTorch master (https://github.com/pytorch/pytorch/pull/47749 3), and you can find docs to use TCP-based init here: https://pytorch.org/docs/stable/distributed.html#tcp-initialization 1