Solved by Philipp_Fischer in post #12 
              
              
                Solution: 
Verified that this is a bug inside cudnn 8.0.1 via official bug report. 
Building torch from source with cudnn 8.0.2 fixes the problem.