Solved by mrshenli in post #2 
              
              
                Hey @yijing 
The message will directly send from  10.0.0.2 to 10.0.0.3. 
In init_process_group, the init_method=“tcp://10.0.0.1:8888” is only for rendezvous, i.e., all process will use the same ip:port to find each other. After that communications don’t need to go through master. 
BTW, if you are us…