cuDNN can still be used, if torch.backends.cudnn.enabled = True.
If you don’t want to use cudnn, you should set this flag to False to use the native PyTorch methods.
When cudnn.benchmark is set to True, the first iterations will get a slowdown, as some internal benchmarking is done to get the fastest kernels for your current workload, which would explain the additional function calls you are seeing.