As a follow-up, there is ongoing effort to integrate a sparse solver natively in torch so this workaround shouldn’t be necessary anymore in the future:
  

      github.com/pytorch/pytorch
  

  
    
  
	  
  

  
    
      cuSOLVER backend for Sparse CSR direct linear solvers
    

    
      
        opened 
        
          
        
        Dec 7, 2021
      
      


      
        
          
          IvanYashchuk
        
      
    

    
        
          module: sparse
        
        
          module: cuda
        
        
          triaged
        
        
          module: linear algebra
        
    
  


  
    ## 🚀 Feature

cuSolverSP (part of cuSOLVER) provides linear solver, least-squa…re solver routines for matrices in CSR format. We can use it as a backend for `torch.linalg.solve` and `torch.linalg.lstsq`.

We are working on improving coverage of Sparse CSR matrices in PyTorch. Solving systems of linear equations is one of the fundamental operations in computational linear algebra.

From cuSOLVER docs:
> The high-level API is designed for ease-of-use, so it allocates any required memory under the hood automatically. If the host or GPU system memory is not enough, an error is returned.

Unfortunately, it means these functions do not allow passing externally allocated buffers, and paths with inputs on GPU are not implemented (there remarks for relevant functions "only CPU (Host) path is provided").

- [ ] `torch.linalg.solve`
    Use [`cusolverSp<t>csrlsvlu`](https://docs.nvidia.com/cuda/cusolver/index.html#cusolver-lt-t-gt-csrlsvlu). 

- [ ] `torch.linalg.lstsq`
    Use [`cusolverSp<t>csrlsqvqr`](https://docs.nvidia.com/cuda/cusolver/index.html#cusolver-lt-t-gt-csrlsqvqr)



cc @nikitaved @pearu @cpuhrsch @IvanYashchuk @ngimel @jianyuh @mruberry @walterddr @xwang233 @Lezcano