Normally when you are doing common operations such as matrix multiplication, matrix-vector multiplication, elementwise multiply-add, convolution, etc., these are dispatched to precompiled kernels in ATen (PyTorch’s tensor operator library), cuBLAS, or cuDNN. No compilation to PTX etc., will happen unless you are doing some kind of JIT compilation (e.g., TorchScript — PyTorch 1.9.1 documentation).
Rather than Python code being “converted,” usually what is happening is the Python code is calling into C++ code that calls the relevant CUDA kernels.