What’s the size of your tensor, i.e. what are B, N, C, C?
How have you benchmarked?
Generally, the linear algebra algorithms have a harder time to exploit the GPU parallelism than many other tasks as the direct methods 7 are fairly sequential.
Best regards
Thomas