Seungsu:

I know that if I set benchmark=true, it will automatically find the right algorithm for my hardware.
But i’m not sure what the algorithms does…


cudnnFind will use different kernel implementations for the current workload, e.g. the current conv layer. You can imagine it as iterating over a conv implementation using matrix multiplications, an FFT, Winograd, etc. Each of these different algorithms (for the same convolution with the given input shapes, padding, dilation, etc.) will be executed and the time will be captured. Once this is done, the fastest would be selected in the best case.