Solved by rahulvigneswaran in post #25 
              
              
                @lynnea718 
The following works properly. 
import os

os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   
os.environ["CUDA_VISIBLE_DEVICES"]=<YOUR_GPU_NUMBER_HERE>

Usually GPU numbers start from 0. Since you have issue with the default device, try anything other that 0.