I faced the same problem when installing Pytorch in a new computer today. I solve it by edit the ‘.bashrc’, please make sure the environment variables of CUDA is set accurately as:
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64/:$LD_LIBRARY_PATH"
export CUDNN_PATH="/home/yuhang/dependencies/cudnn-5/cuda/lib64/libcudnn.so.5"
The actual path will change based on your setup.
Hope it will helps.