@yaysummeriscoming I’ve checked the code as well, and yes even in the case of depthwise we are still dispatching to thnn depthwise kernels: https://github.com/pytorch/pytorch/blob/51f1c4fea5e066e3cbb7a7251114de4381d2a926/aten/src/ATen/native/Convolution.cpp#L330 26
We haven’t used CuDNN implementation because those kernels aren’t very optimized according to an NVIDIA employee: https://github.com/pytorch/pytorch/issues/1708#issuecomment-384015108 54
Once the situation changes and CuDNN has faster and optimized convolution routines for Depthwise, we will switch to dispatching to CuDNN.
I’ve opened and issue to track this: https://github.com/pytorch/pytorch/issues/15513 157