If you want to benchmark CUDA time add ; torch.cuda.synchronize() at the end of the command. Also, it seems that this first like also had to initialize CUDA (you can see there was only 1 iteration in a loop, because timeit thought it will take forever to run it every time). This means that the timing is not reliable