Solved by eqy in post #4 
              
              
                It should be close to where you specify the address (e.g., MASTER_PORT here Getting Started with Distributed Data Parallel — PyTorch Tutorials 1.8.1+cu102 documentation).