Hi,
Problem:
Based on the issues on Github, PyTorch does not support torch.solve for sparse tensors (neither forward nor backward). The main issue is runtime error: no stride. I wonder is there any workarounds for any special case so I can fix my issue? Based on my experiments, there is no way that I can handle my problem by using dense matrices and also, I need backward compatibility too.
I do not know it is possible, but can I use a third-party libs such as cupy and still maintain computational graph and backward through it?

  
    
    
    Rewriting spdiags & spsolve? 
  
  
    I’m trying to rewrite some legacy code and switch from scipy to pytorch and I’m having difficulties re-writing the spdiags and spsolve. 
How could I perform these operations on GPUs? 
A = spdiags(B.T,d,k,k)

A = A + A.T + spdiags(D.T, 0, k, k)

return spsolve(A, input)
  



  
      github.com
  
  
    Quansight-Labs/rfcs/blob/pearu/rfc0005/RFC0003-sparse-roadmap/SparseSupportState.md#linear-algebra-algorithms 1

This file is auto-generated, do not edit!

# The state of PyTorch tensor layouts support

The following table summarizes the state of PyTorch tensor layouts for
different PyTorch functions from the following namespaces:
- torch
- torch.nn.functional
- torch.sparse

|                              Section                               |                        strided@cpu                        |                      strided@cuda:1                       |                             sparse_coo@cpu                             |                           sparse_coo@cuda:1                            |
| :----------------------------------------------------------------- | :-------------------------------------------------------- | :-------------------------------------------------------- | :--------------------------------------------------------------------- | :--------------------------------------------------------------------- |
| <a href="#tensor-constructors">Tensor constructors</a>             | PASSED: 21, SKIPPED: 3                                    | PASSED: 20, SKIPPED: 4                                    | FAILED: 11, N/A: 2, PASSED: 7, SKIPPED: 4                              | FAILED: 12, N/A: 2, PASSED: 6, SKIPPED: 4                              |
| <a href="#trigonometry-functions">Trigonometry functions</a>       | PASSED: 78                                                | PASSED: 78                                                | FAILED: 71, PASSED: 7                                                  | FAILED: 71, PASSED: 7                                                  |
| <a href="#arithmetics-functions">Arithmetics functions</a>         | NOTIMPL: 1, PASSED: 36                                    | NOTIMPL: 1, PASSED: 36                                    | FAILED: 31, NOTIMPL: 1, PARTIAL: 2, PASSED: 3                          | FAILED: 31, NOTIMPL: 1, PARTIAL: 2, PASSED: 3                          |
| <a href="#linear-algebra-algorithms">Linear Algebra algorithms</a> | PASSED: 32                                                | FAILED: 2, PASSED: 30                                     | FAILED: 28, PASSED: 4                                                  | FAILED: 28, PASSED: 4                                                  |
| <a href="#convolution-operations">Convolution operations</a>       | PASSED: 9                                                 | PASSED: 9                                                 | FAILED: 9                                                              | FAILED: 9                                                              |
| <a href="#matrix-functions">Matrix functions</a>                   | NOTIMPL: 2, PASSED: 24                                    | NOTIMPL: 2, PASSED: 24                                    | FAILED: 24, NOTIMPL: 2                                                 | FAILED: 24, NOTIMPL: 2                                                 |
| <a href="#general-array-functions">General array functions</a>     | PASSED: 57                                                | PASSED: 57                                                | FAILED: 38, PASSED: 19                                                 | FAILED: 38, PASSED: 19                                                 |


  This file has been truncated. show original

  
  
    
    
  
  



  
      docs.scipy.org
  
  
    

scipy.sparse.linalg.spsolve — SciPy v1.5.4 Reference Guide 2



  
  
    
    
  
  



  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      The state of sparse Tensors 4
    

    
      
        opened 
        
          
        
        Jul 21, 2018
      
      


      
        
          
          weiyangfb
        
      
    
  



  This note tries to summarize the current state of sparse tensor in pytorch. It describes important invariance and properties of sparse...



    module: sparse
    triaged


  
  
    
    
  
  



  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Todo functions and autograd supports for Sparse Tensor
    

    
      
        opened 
        
          
        
        Jun 25, 2018
      
      


      
        
          
          weiyangfb
        
      
    
  



  Here summarizes a list of requested Sparse Tensor functions and autograd supports from previous PRs. Please feel free to comment on...



    module: sparse
    triaged


  
  
    
    
  
  


So far:
I have found that Tensorflow also does not like sparse operations with the same reasons as PyTorch.
Apparently, there are some libraries such as cuSolver 14 which supports forward but still backward needs to be implemented. As there are no python bindings for this lib, I wonder does it make sense to make this happen in PyTorch.
Another approach which I don’t like (nothing is like PyTorch  ) is that change framework to Jax as apparently it supports forward/backward for spsolve or cholesky_solve, or even using another language such as Julia.
Please let me know if there is any ongoing project on solving this issue or any other ideas.
Bests