Hi,
pytorch has done a great job for normal dense tensor, and I have used it for not only deep learning applications, but also some basic gpu boosted inference task, such as kmeans and tsne.
However, currently the sparse tensor support is not so good. According to my observation, only matrix multiplication and element-wise math operation is supported, while other important functions such as slice and reduce sum is missing.
This cause a lot trouble when we want to use pytorch’s sparse tensor to handle some inference jobs.
Here I have a small project which present a sparseTensor wrapper for slice and reduce sum based on pytorch dense tensor, sparse tensor and some cupy kernels(only for inference, since no graph registration is handled)

  
      
      GitHub
  
  
    

overshiki/sparseTensorUtils 10

sparseTensorUtils - sparse tensor utils based on pytorch tensor, provide a class:sparseTensor which support slice, reduce sum, element-wise math operation and so on


  
  
    
    
  
  


If you guys have a plan to implement these features with c code, that would be excellent!  since GPU memory is always a big problem when we want to handle some big data task.  Would you like to do this soon? Or any long-term plans?
Thanks a lot.
Best