swpd:

Is there a way we can explicitly tell cudnn to select a faster kernel instead of the bad one?


No, there is no user facing API to do so.



 swpd:

Have you tried this script on cudnn8 without setting torch.backends.cudnn.benchmark = True ?


Yes, ran it with the default setup first and compared it to benchmark mode, which didn’t yield a speedup.



 swpd:

By the way, do you know what does the kernel cudnn::detail::dgrad2d_alg1_1 do? I searched Google but couldn’t get any useful info.


It’s a cudnn “data gradient” kernel, which computes the gradient for the input activation.