Solved by mrshenli in post #4 
              
              
                Since the IP was an issue, I assume you will train the model using multiple machines? If yes, the --nnode and --node_rank also need to be configured, see: https://github.com/pytorch/examples/tree/master/distributed/ddp 


Since each process can see two devices (CUDA_VISIBLE_DEVICES=0,1), you will …