combining GPUs automatically is not easy to do. PyTorch does not do this implicitly.
Hence, you see that regardless of how many GPUs you have, the SVD is running only one one GPU and going out of memory.