I can’t help you much, but PyTorch can be compiled with Magma 13 which is a linear algebra library for GPU and includes QR, SVD, eigenvalues and BLAS functions (matmul, transpose)
I am not sure where/how it’s used though. — Edit: there 18

image.jpg850×677 210 KB