Solved by mrshenli in post #2 
              
              
                What does this mean? Their IPs are not reachable from each other? 

I have seen  nn.DistributedDataParallel  but how do I mention the IP address of multiple servers? 

If they can reach each other through network, yes, DistributedDataParallel can work across multiple machines. You need to provide t…