To reduce the number of variables at play here, let’s maybe start by trying to get the ProcessGroup backend (based on Gloo) to work. (I’m saying this because the TensorPipe backend also uses Gloo internally and thus could fail even if the error didn’t come from TensorPipe directly). Could you try with that backend and see what happens?
Also, I didn’t understand what your system admid did. Did they open up some range of ports? Or maybe just port 49552 that we were mentioning before? Or did they remove the firewall entirely? Only the latter option will work, because the former ones will be ineffective since both Gloo and TensorPipe will use different ports each time they run.
What is more, in fact, is that Gloo opens many listening sockets, one for every peer that it needs to communicate with. Hence having a fixed range of ports (or even just a single one) that Gloo is allowed to  use won’t scale.
One last thing: the “connection closed by peer” error seems to suggest that the nodes were initially able to connect but that one of them then crashed (although it could also have other explanations). Were you able to see whether the other node had some different error in its log (which may be the real root cause)?
One way you may be able to circumvent all this, and not require the intervention of your sysadmin, is by setting up some sort of VPN between your two nodes so that they can open whatever ports they like but it would still all pass through the firewall properly. I do not really know how to set such a thing up, and it may require you to use Docker if you need to avoid root privileges. In the end it may get complicated.