Thanks again for your reply!
So I don’t want to bypass cudnn, because I still want it to get that full GPU speed-boost.
But I want to figure out where the cudnn LSTM is built, so that I can apply my modifications there.
Or is there no way to change how cudnn builds the LSTM?
Interestingly though, I tried your suggestion (torch.backends.cudnn.enabled = False) and it works!
My modifications take effect, and It’s about 5 times faster than without Cudnn (1 min vs 5 min per epoch).
However, it’s also about 3 times slower than using the cudnn backend (20 sec per epoch).
So if there’s a way to modify the architecture of the LSTM in the cuda backend, that’d be the best option.
I’m gonna be running this network for 100 epochs 60 times, so I need every little GPU boost I can get.
Regardless, even if this is the best you can do for me, THANK YOU, you probably saved me hundreds of hours.