I have the same issue as @ddhruvkr:
I have pytorch 1.1, cuda 9.0, cudnn 7, ubuntu 18.04, and a RTX 2080ti. When I load my model with .cuda(), I get cuDNN error: CUDNN_STATUS_EXECUTION_FAILED.
And the model loads without error when I run the code a second time.
There is a warning before the stacktrace (that I missed somehow):
UserWarning: 
    Found GPU0 GeForce RTX 2080 Ti which requires CUDA_VERSION >= 10000 to
     work properly, but your PyTorch was compiled
     with CUDA_VERSION 9000. Please install the correct PyTorch binary
     using instructions from https://pytorch.org

Soumith said here 19:

CUDA 9 and RTX 2080 Ti simply aren’t compatible and dont play well togethere.
An older CuDNN version working is likely a side-effect rather than expectation.
Use CUDA10 and CUDA10 versions of CuDNN etc. for RTX 2080 which is Turing architecture

Make sure that the CUDA version is 10.