I see. Technically, it shouldn’t be too hard to let the ProcessGroup RPC backend to use MPI, as it only requires its send/recv features. One options could be adding one field to ProcessGroup RPC backend construction time options, and let users decide whether they want to use Gloo, or NCCL (> 2.7), or MPI.
cc @lcw any thoughts on MPI + TensorPipe?  Does it make sense to add MPI as a channel type for TensorPipe?