Can I control exactly when MXNet should run performance tests?
I have an MXNet application that is about to go to production. It is running ok, but it has a variable batch size and it causes MXNet to perform autotune often.
Although the batch size is variable, it is typically 1. I'm fine with performance loss on bigger sizes (it is still better than interrupting it all the time to autotune).
If I disable autotune with export MXNET_CUDNN_AUTOTUNE_DEFAULT=0 the network runs considerably slower.
So my question: is there a way to run autotune only once, preferably at my call?
There is currently no way to run autotune only once. When autotuning is enabled MXNet will run performance tests to find the best algorithm to run for Convolution/DeConvolution operators. The best algorithm is cached with the specific input shape, output shape. So as long as you have the same input/output shape, the performance tests should not re-run. However, if the shape changes then it would trigger the algorithm to re-run.
MXNet CuDNN Autotune management