Is this about uneven inputs on different processes? See:

https://github.com/pytorch/pytorch/issues/33148 79
https://github.com/pytorch/pytorch/issues/38174 47

If all processes know when to exit, simply break the loop would work. The tricky case is when one processes breaks the loop but other processes proceed as mentioned in the above two issues.