Could you post the code you’ve used to time these operations?
Note that CUDA operations are asynchronous in PyTorch, so that you should synchronize before starting and stopping the timer:
torch.cuda.synchronize()
t0 = time.time()
# your operation
torch.cuda.synchronize()
t1 = time.time()

If you didn’t add these sync points, you might just time the initial CUDA context creation etc.