Comparing the FFT execution time with the one of Conv2d, I noticed that FFT operation is way slower than expected.
Doing some investigation, I noticed the Conv2d uses the FFT method, and somehow is able to launch several stream.
While when I called nn.funcitonal.fft.fft2(inputs) myself, I was only able to launch one stream.
Is there any way to improve the FFT calls? I guess it is safe to say it should be possible. cudnn_convolution is somehow doing it.
How does the number of stream affects speed?

This is the profiling of nn.functional.conv2d

Screen Shot 2021-09-24 at 1.10.23 PM1988×1125 344 KB

This is the profiling of nn.functional.fft.fft2

Screen Shot 2021-09-24 at 1.39.39 PM2032×1169 190 KB