Solved by charan_Vjy in post #4 
              
              
                I had actually missed the nvidia-smi output: 
It shows that 2779MiB / 6144MiB (46%) is being utilzed. This implies that the model was successfully loaded into the GPU. One empirical way to verify this is to time it using device = 'cpu' and then time it using device = 'cuda' and verify the different …