Backprop Through sparse_dense_matmul
Do You have a roadmap for the whole sparse module? I’m curious if you’re planning to use cuSPARSE and if there is any way to contribute to this module. I’d need a backward pass for sparse_mat @ dense_mat. What are my options? Can I write a Function using the available spmm for forward, and add some quick-and-dirty cython code to do the backward? This would mean returning a sparse grad, is this supported?
elanmart
ebetica
Hi, for now sparse tensor support basically evolves with the needs of our respective projects. Basic GPU support is being worked on - it will rely on cuSPARSE for some operations. returning sparse gradients from backward should work in addition to the thread @ebetica pointed to, since pytorch supports hybrid tensors (i.e. tensors with both sparse and dense dimensions), we may add a sparse * dense -> hybrid function in the future (where “hybrid” here means one sparse dimension and one dense dimension). That would probably be more efficient in this case.
martinraison
elanmart