Hi,
I’m getting an CUDNN_STATUS_INTERNAL_ERROR error like below.
python train_v2.py
Traceback (most recent call last):
  File "train_v2.py", line 113, in <module>
    main()
  File "train_v2.py", line 74, in main
    model.cuda()
  File "/home/ahkim/Desktop/squad_vteam/src/model.py", line 234, in cuda
    self.network.cuda()
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 249, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
    module._apply(fn)
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
    module._apply(fn)
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
    module._apply(fn)
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 112, in _apply
    self.flatten_parameters()
  File "/home/ahkim/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 105, in flatten_parameters
    self.batch_first, bool(self.bidirectional))
RuntimeError: CUDNN_STATUS_INTERNAL_ERROR

What should I try to resolve this issue?
I tried deleting .nv but no success.
The same code runs without error using Nvidia  Driver Version: 396.26 (cuda V9.1.85. torch.backends.cudnn.version(): 7102). I’m getting an error using Driver Version: 390.67 (cuda V9.1.85. torch.backends.cudnn.version(): 7102)