LOL, very interesting.
For numpy, here is a post about how to disable the OpenBLAS affinity resetting behavior. https://stackoverflow.com/questions/15639779/why-does-multiprocessing-use-only-a-single-core-after-i-import-numpy 7
But, even with this, my issue still persists with PyTorch.