If this is an inference task without the need to do things like broadcasting gradients, could you “outsource” the CPU inference to a subprocess subprocess — Subprocess management — Python 3.10.2 documentation 3 e.g., via POpen as a workaround?