Solved by ptrblck in post #6 
              
              
                Thanks for sharing this output! 
I was able to reproduce a minor regression in the 1.10.0+cu113 wheels, which seems to be resolved already using the latest cuDNN release (8.3.0.96). 
Code used to profile the workload: 
random.seed(2021)
np.random.seed(2021)
torch.manual_seed(2021)

print(torch.cuda.…