I ran into the Runtime Error: CUDNN_STATUS_NOT_INITIALIZED when I was training a model that has cnn.
I checked my installation
>>> import torch
>>> print(torch.backends.cudnn.is_acceptable(torch.cuda.FloatTensor(1)))
True
>>> print(torch.backends.cudnn.version())
7005


One thing that might cause the problem but I’m not sure:
On the server that I’m using:
(test_cudnn) zhangjuexiao@ubuntu:/data/disk1/private/zhangjuexiao/allennlp$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44

cat /usr/local/cuda-8.0/include/cudnn.h | grep CUDNN_MAJOR -A 2

#define CUDNN_MAJOR      6
#define CUDNN_MINOR      0
#define CUDNN_PATCHLEVEL 21


But I install this in my virtual environment:
conda install pytorch torchvision cuda90 -c pytorch

I’ve been tortued by it for days…
How to solve the problem? Any kind help is appreciated!