Solved by Noa_Lubin in post #2 
              
              
                Solution by Yanai Elazar: 
You can define an environment variable like this: 
CUDA_VISIBLE_DEVICES=1 
This way, only this gpu will be available for the running program, and you won’t leak into other gpus. This way in the code you need to run on a single gpu, and not specify one specifically.