Thank you for the quick reply. For some context, I am trying to launch several agents inheriting mp.Process playing in separate environments, each enqueuing frames in a shared Queue. A single prediction subprocess is dequeuing batches of frames, does an inference step through the neural network estimator and passes back actions to each agent.
I am already seeing nice speed-ups compared to the naive asynchronous implementation and after I changed the start method to spawn it seemed to be running fine on my laptop with cuda support (shaving about 20% of the execution time when using cuda). However when I tested it on our servers I got a RuntimeError: CUDNN_STATUS_NOT_INITIALIZED.
I get now it’s not really trivial setting this up and I will continue working on the implementation leaving cuda aside for now. I do plan to revisit the issue and I’ll post here about any developments.