It’s not an “issue”, but just how asynchronous execution works.
The wrong timings are created since the CPU can launch the CUDA kernels and continue executing the host code until a synchronization is needed. In the meantime the GPU is working in the background.
If you thus start the timer, launch the kernel, and directly stop the timer afterwards, the GPU might stil be busy and you would not profile the actual GPU workload.
As already described, you need to synchronize the code to be able to profile the kernel timings on the GPU or the better way would be to use torch.utils.benchmark, which was created for use cases like these.