Nikronic:

What could be the possible explanation for this?


As @albanD pointed out, the memory access pattern is beneficial in NHWC for (some) convolution implementations. Also, TensorCores natively use NHWC tensors, so you are saving a permutation inside the cudnn kernels.
The last slide in this presentation 145 shows some numbers.