When choosing CUDA device, you can detrmine the GPU by doing torch.cuda.set_device([device]). Read more here; link 1.
Feel free to read this thread 1 too for alternative options such as setting CUDA_VISIBLE_DEVICES

  
    
    
    How to change the default device of GPU? device_ids[0] 
  
  
    How to change the default device of GPU? 
for some reason ,I can not use the device_ids[0] of GPU, 
I change the following code:（in data_parallel.py） 
if output_device is None:
output_device =device_ids[0]

to 
if output_device is None:
output_device =device_ids[1]

but it still seem to used the device_ids[0] 
all tensors must be on devices[0]？ 
How to change it？