ragnaarok:

As I said, It seems that my training is more stable after I changed some tweaks in my model so I guess I just had an unstable architecture.


I don’t think the error was caused by the architecture, but as @tom mentioned, might be a faulty kernel in cudnn, which should be fixed. In case you have an executable code snippet or run into this error again, could you please post it here?