Hi here is the patch, https://github.com/pytorch/pytorch/pull/2903 67. Feel free to try it out and let us know if you have any questions.
Alternatively, if you have IPoIB setup, it should be easier to use IPoIB ip address directly without re-compilation. In that way you can also utilize the IB link.