Hmm yeah that will likely use make cuDNN pick a different algorithm. I don’t think it’s optimized for 3x300, so it probably falls back to a slow gemm implementation, while 1x3 is quite commonly used, and will likely use a smarter algo.