Based on the output of nvidia-smi you are initializing a CUDA context in each worker. I don’t know how you are spawning them but you might want to check if you are (accidentally) initializing it or could also try to mask the device in the workers.