@CDhere For benchmarking, should probably place a torch.cuda.synchronize(device="cuda:0") before you print. Also, the channels_last speedups for convs are most relevant for float16. You might want to set a, b and the convs dtype=torch.float16
If there is odd float32 behaviour like you see, could be a regression in 1.7 builds, I saw several issues that have been fixed in more recent NGC (20.12) container releases that build against newer versions of cuDNN/CUDA.