Solved by H-Huang in post #2 
              
              
                Thanks for posting your code and details! 
I see that when you init the process group with NCCL you specify the init_method as “tcp” and provide a local IP and port. Can you ensure that those are reachable? Alternatively, since you are already setting the address and port as environment variables an…