You could check what exactly is fused in the operation by printing the graph via .graph_for for this module. This should show the fused operations and would give you more information about the difference in your timing.
I also assume you’ve written a custom CUDA implementation or are you comparing the GPU kernel vs. your C++ (host) code?