If everything is indeed a cuda tensor, then the computation should be happening on GPU. You can run nvprof e.g., nvprof python myscript.py to get a sense of which kernels are being executed on GPU.
Alternatively if you are running the script interactively (or it is long-running enough), you can do something like watch nvidia-smi to get a sense of GPU usage as the script is running.