It’s not a GPU specific problem, we’re probably just using different cuDNN calls to select algorithms, and for some reason they fail to pick the fast ones for this use case. I don’t know if there’s no other way, it will take me some time until I properly benchmark it and see what’s the issue. If you want to use it now, just reshape the input.