Hi!
With the help of the one and only @ptrblck : this has come up before and it seems that it is mostly a profiling artefact:

  

      github.com/pytorch/pytorch
  

  
    
  
	  
  

  
    
      Excessive call to cudaGetDevice and  cudaSetDevice 1
    

    
      
        opened 
        
          
        
        Mar 15, 2019
      
      

        
          closed 
        
          
        
        Mar 18, 2019
      
        

      
        
          
          omry
        
      
    

    
    
  


  
    ## 🐛 Bug

While profiling my code, I noticed millions of calls to cudaGetDevic…e which were slowing down my 
code significantly (it's an RL use case and the model I am using is small).
I have a minimal repro that demonstrate that a single forward call on nn.Linear() is triggering 11 cudaGetDevice() calls and 3 cudaSetDevice() calls.

## To Reproduce
See attached code snippet.

Steps to reproduce the behavior:
/usr/local/cuda/bin/nvprof --profile-from-start off python  cudagetdevice.py
Output:

```
$ /usr/local/cuda/bin/nvprof --profile-from-start off python  cudagetdevice.py 

==112656== NVPROF is profiling process 112656, command: python cudagetdevice.py
==112656== Warning: Profiling results might be incorrect with current version of nvcc compiler used to compile cuda app. Compile with nvcc compiler 9.0 or later version to get correct profiling results. Ignore this warning if code is already compiled with the recommended nvcc version 
==112656== Profiling application: python cudagetdevice.py
==112656== Profiling result:
            Type  Time(%)      Time     Calls       Avg       Min       Max  Name
 GPU activities:   73.95%  5.0880us         1  5.0880us  5.0880us  5.0880us  sgemm_32x32x32_NT_vec
                   26.05%  1.7920us         1  1.7920us  1.7920us  1.7920us  _ZN2at4cuda75_GLOBAL__N__51_tmpxft_00005244_00000000_11_Copy_compute_75_cpp1_ii_dd3fb9a321kernelPointwiseApply2IZN75_GLOBAL__N__51_tmpxft_00005244_00000000_11_Copy_compute_75_cpp1_ii_dd3fb9a36CopyOpIffE5applyERNS_6TensorERKS6_EUlRfRKfE_ffjLi1ELi2ELi1EEEvNS0_6detail10TensorInfoIT0_T2_EENSF_IT1_SH_EESH_T_
      API calls:   99.99%  1.03481s         2  517.41ms  12.000us  1.03480s  cudaLaunchKernel
                    0.01%  67.541us        11  6.1400us  4.5000us  13.500us  cudaGetDevice
                    0.00%  14.583us         3  4.8610us  4.5000us  5.5830us  cudaSetDevice
                    0.00%  9.0410us         2  4.5200us  4.5000us  4.5410us  cudaGetLastError
```

## Expected behavior
Number of calls to cudaGetDevice should be minimal, I would argue that it should not be called at most once, if that.

## Environment

PyTorch version: 1.0.1.post2
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 16.04.6 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: Quadro P5000
GPU 1: Quadro P5000

Nvidia driver version: 418.39
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.15.4
[pip] torch==1.0.1.post2
[pip] torchfile==0.1.0
[pip] torchvision==0.2.2
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.1                      144  
[conda] mkl_fft                   1.0.6            py36hd81dba3_0  
[conda] mkl_random                1.0.2            py36hd81dba3_0  
[conda] pytorch                   1.0.1           py3.6_cuda10.0.130_cudnn7.4.2_2    pytorch
[conda] torchfile                 0.1.0                     <pip>
[conda] torchvision               0.2.2                      py_3    pytorch


 - PyTorch Version (e.g., 1.0):
1.0.1
 - OS (e.g., Linux): Ubuntu 16.04
 - How you installed PyTorch (`conda`, `pip`, source):
pip inside a conda env
 - Build command you used (if compiling from source):
 - Python version:
3.6
 - CUDA/cuDNN version:
10.0
 - GPU models and configuration:
Quadro P5000
  

  

  
    
    
  

  


Of course, you could re-run the experiment of replacing all calls to cudaGetDevice/cudaSetDevice with a noop and see if things have changed.
Best regards
Thomas