I have not enable cudnn.benchmark in my code, but i dont know if it can be enabled by default or not.
my input shapes change a lot from 600600 to 12001200. and i can see that the code suddenly become slow, i havenot noticed this kind of slowness before, so i assume this can be caused by cudnn, but im not 100% sure yet