If you are masking the GPU via CUDA_VISIBLE_DEVICES=1 your script will only see a single device as cuda:0 which would be GPU1 in this case. You thus have to use cuda:0 inside the script and will see the expected invalid device ordinal error is you are trying to use any other cuda:id.