Solved by ptrblck in post #2 
              
              
                CUDA operations are executed asynchronously, so you would have to synchronize the code before starting and stopping the timer via torch.cuda.synchronize(). Otherwise you’ll only profile the Python overhead as well as the kernel launch in the first iterations until your script encounters a blocking o…