Solved by zcajiayin in post #2 
              
              
                With pytorch 1.6, now you can call torch.sparse.mm(A, B) where A is the sparse matrix and B is a dense matrix (I think it also work with sparse B).