Solved by ptrblck in post #2 
              
              
                Most likely the CUDA context is also initialized on the default device. 
You could mask all devices and only pass the wanted one via: 
CUDA_VISIBLE_DEVICES=1 python script.py args