Solved by ptrblck in post #7 
              
              
                Thanks for the update! 
I’ve added some missing warmup iterations, since you are using cuDNN’s benchmark mode, so the first iteration will see a slowdown due to the internal profiling. 
I can reproduce the slower kernel execution time in FP16 using the default memory format, but see a speedup using m…