ALso you can search for old installations like this find . -name libcudnn* 2>/dev/null 
For example I was having results for

find . -name libcudnn*8.0.5* 2>/dev/null 
find . -name libcudnn*8.1.0* 2>/dev/null 

Then I just rm -R /usr/local/cuda-11.1.0 and let 11.2.0 and it found the correct library.