HI, @apaszke
There is cudnn5.0 lib on my PC, however I got warning:
UserWarning: PyTorch was compiled without cuDNN support. To use cuDNN, rebuild PyTorch making sure the library is visible to the build system.
  "PyTorch was compiled without cuDNN support. To use cuDNN, rebuild "

How to build pytorch with cuDNN support?
cudnn.h is in /usr/local/cuda-8.0/include/cudnnv5/ and cudnn.so.5 is in /usr/local/cuda-8.0/lib64/cuDNNv5/. The path has been added in system environment variable.
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64/cuDNNv5:$LD_LIBRARY_PATH

I build the pytorch from source code.
cd pytorch-root/ & python setup.py install