I replace process group with tensorpipe, I also meet
...
RuntimeError: [/pytorch/third_party/gloo/gloo/transport/tcp/pair.cc:184] listen: Address already in use

It seems that tensorpipe still depends on gloo.
I run init_process_group and all_reduce, and the error is the same.