Referencing corresponding bug: Pytorch autograd sometimes fails with CUDNN_STATUS_INTERNAL_ERROR · Issue #52263 · pytorch/pytorch · GitHub 11
As stated should be fixed with torch 1.8 and cudnn 8.1.
I’m testing it and will update.