Solution by Yanai Elazar:
You can define an environment variable like this:
CUDA_VISIBLE_DEVICES=1
This way, only this gpu will be available for the running program, and you won’t leak into other gpus. This way in the code you need to run on a single gpu, and not specify one specifically.