That’s expected as cuDNN often leads to some significant speedup.
As I don’t have an RTX card myself, I’m just guessing that a version mismatch might throw your error.
Which CUDA version are you using?
Maybe @ngimel might help here.