Thank you for your reply and sorry for the delayed response!
I have tried to mask the device in the worker processes, in their init method of the worker processes, by setting “CUDA_VISIBLE_DEVICES”="" in the environment, but this has also not worked. I have also checked for accidental references to cuda without a finding any.
I produced a minimum working example of the problem in this repo: GitHub - amarildolikmeta/torch_workers_gpu 1
It requires open ai gym (it includes a simple test environment).
The main script initializes the worker processes then runs a loop alternating between the main process running some episodes in the gpu, followed by 2 workers processes running some episodes with the models in the CPU.
This minimum example reproduces this behaviour in my ubuntu 18.04, with python 3.6 and torch 1.10