You could check the Performance Guide and enable e.g. torch.backends.cudnn.benchmark =True in case you are using static input shapes so that cuDNN could profile all kernels and select the fastest one for your use case.