Thanks for this information. Could you compare the script with PyTorch 1.7.0 and CUDA10.2, which uses cudnn7.6.5? I still think your 2080Ti is seeing a regression on cudnn8.0.3.
I can run the workload on my system later with cudnn8.0.5 and the latest internal version.