It would be great if Pytorch could distinguish an out-of-memory error from the other stuff that causes a CUDNN_STATUS_EXECUTION_FAILED. Is there any ongoing effort in this matter?
A better error message would be particular useful, for instance, when doing neuro architecture search, and you want to automatically reduce the batch size, if an OOM error is detected.