You don’t need to use subprocesses. Cuda calls are asynchronous.
Besides you don’t need to use set_device
You can allocate the model and inputs by choosing the device when doing .cuda(idx) or .to('cuda:idx')
If you really want to do that, you have to use daemon threads.