Solved by justusschock in post #2 
              
              
                If you set CUDA_VISIBLE_DEVICES = 1  your python script is only able to “see” this GPU and thus refers to it as “cuda:0” . If you start it on cuda:0 with CUDA_VISIBLE_DEVICES being set to 1 and afterwards use nvidia-smi you should see your model running on GPU1