Could you share the code snippet you’ve used to profile this operation?
Note that CUDA operations are executed asynchronously, so you would have to synchronize the code before starting and stopping the timer (or use the torch.utils.benchmark utilities, which will synchronize the code for you).