Hey @yijing
The message will directly send from  10.0.0.2 to 10.0.0.3.
In init_process_group, the init_method=“tcp://10.0.0.1:8888” is only for rendezvous, i.e., all process will use the same ip:port to find each other. After that communications don’t need to go through master.
BTW, if you are using p2p comm, torchrpc 3 might be useful too. Here 2 is a tutoral.