Solved by tom in post #4 
              
              
                Do you run your PyTorch model in Python or have you JITed it to TorchScript? 
We discuss this a bit in Chapter 15 of our book, if you run your model through the JIT - even from Python - you avoid the infamous Python GIL for all intermediates, which may give you an edge in multithreaded deployment sc…