On the CPU, we use Intel MKL to ship with our binaries. ATen can use other Linear Algebra libraries as well, that expose a LAPACK interface, such as OpenBLAS, Eigen.
On the GPU, we use cuBLAS, cuSolver and MAGMA 41