Yes, after setting torch.backends.cudnn.benchmark = True  at the beginning of the script, I got ~2.0008ms. Thank you for sharing this!
However, the model I use accept variable input instead of fixed one, so this result in searching the fastest every time it meets a different shape of input.

The fast kernel seems to be already selected in the upcoming cudnn8 release.

By the way, which version of PyTorch is packed with cudnn8?  The latest stable version 1.5.1 comes with cudnn7.6.5.