Hi!!!
Did you figure out how to solve this? I’m having a similar issue.
The first time (or couple of first times) I run a script it works fine, but afterwards it will throw either the same error as yours (CUDNN_STATUS_INTERNAL_ERROR) or a Cuda runtime error (4): unspecified launch failure or segmentation fault (core dumped).
I supose there’s something wrong with my cuda installation but I’ve tried reinstalling it several times, I’ve even tried switching back to kernel 10.3 instead of 13.1 and reinstalling it but so far I cannot get rid of this weird behavior.
I’ve also tried building pytorch from source and switching to pytorch 0.1.12.
I supose you used different versions than mine, but any pointers would be appreciated. I’m running out of ideas.
Btw I’m using Nvidia’s driver 384.69 and Cuda 8.0.61, both installed with the runfiles for Ubuntu 16.04.3 with kernel 13.1 on a system with a single Gtx 1080 and a ryzen 1600 processor.