Solved by eqy in post #2 
              
              
                If everything is indeed a cuda tensor, then the computation should be happening on GPU. You can run nvprof e.g., nvprof python myscript.py to get a sense of which kernels are being executed on GPU. 
Alternatively if you are running the script interactively (or it is long-running enough), you can do …