I answered your issue on github torch.distributed and subprocess do not work together? · Issue #62381 · pytorch/pytorch · GitHub 2. Hopefully that clarifies how to use init_process_group. But I suspect this may not address the original issue brought up in this post? It is hard to say why my_eval_code.py is blocking and what exactly that code is doing. If you can find a locally reproducible example where it blocks that would help!