Hey @mbehzad, I noticed you are using NCCL. IIUC, NCCL is not available on Windows. Could you please try gloo instead?
Another question is that which PyTorch version are you using? In v1.7.*, the distributed package only supports FileStore rendezvous on Windows, TCPStore rendezvous is added in v1.8.