Note that CUDA operations are asynchronously executed, so you would need to time the operations in a manual loop and use torch.cuda.synchronize() before starting and stopping the timer.
Also, the workload is tiny, thus the kernel launch times might even be larger than the actual execution time.