Hi,
Yes I just remembered, that there was a bug in some handling of cudnn algorithm.
Basically cudnn choose the fastest possible algorithm but that sometimes require more memory than what you have. This should be handled properly and select a less memory hangry algorithm. There was a bug in this fixed in this pr 10.
Not sure if it made it into 1.0 but I think it did. Enabling benchmark mode should fix the issue and keep the best runtime !