MPI backend isn’t that well-tested, and depending on what MPI version you are using, the thread-support in MPI also has known perf issues and being not well-tested. PyTorch’s MPI backend currently uses MPI threads. That’s why we recommend to use Gloo backend.
NCCL2 is another backend we are currently actively working on to support IB.
https://github.com/pytorch/pytorch/pull/3435 130 is PR and you can check the status there. It will be available hopefully soon.