Spawning/forking without an exec in general is always tricky and can introduce subtle bugs (like you had in this case) We do not have a mechanism to detect double initializations and it is unlikely that we can come up with a solution that works for everyone considering the size of our user base.
My only suggestion would be to rethink how you import and initialize your Python modules. Ideally only a single subprocess should import the relevant packages from PyTorch while the rest performs other auxiliary tasks.

Now I guess I get the reason why this module was originally provided only as a script.

That is correct. The internals of how the launcher script works is still under development. So I strongly suggest using the script if possible. We do not guarantee any backwards compatibility for our API since it is not public yet.

Thank you so much everyone!

You are welcome! Good luck with your experiments and let us know if you need any other help!