Hello Dick!



 rpfeynman:


A is sparse, so A-transpose is too. (Supposedly pytorch
knows this, but maybe not.)

I think you have find the culprit here: something weird happens to sparse matrices when you transpose them.
…
The result I get is the following
Matrix-vector product (sec)0.3793952465057373
Residual calculation + masking (sec)0.0005414485931396484
Transpose calculation (sec)1.4321186542510986
Matrix (A transpose)-vector product (sec)24.383739471435547

First of all: taking the transpose takes waaaaaay too much time than it should.
Then, calculating the matrix product using the transposed matrix is what takes forever.


Those timings do look suspicious.
Your A-transpose . v timing is again about 100 times longer
than your A . v timing.  This is in line with the backward / forward
timing discrepancy you reported earlier.  And I agree that the
timing for taking the transpose also seems long.
It’s almost as if taking the transpose turns the matrix into a
full matrix (but your comment about memory suggests that
this is not the case).
I did a quick test with a (small) torch.sparse.FloatTensor.
Taking the transpose (torch.t()) resulted in a sparse tensor,
so my simple-minded theory about the transpose being a full
matrix looks unlikely.  I think it’s a long shot, but could you print
out the type of your At variable just to make  sure that it’s still
sparse (and that nothing else fishy is going on)?


Maybe we should raise an issue on github about it?


A github issue would seem appropriate.  Or perhaps some
pytorch experts – who seem somewhat sparse around here
sometimes – could chime in on this thread with any ideas
about what might be going on.
I do think what you’re doing should work.  I can’t think of any
impediment to using the much faster sparse-matrix manipulations
with A-transpose (or with the back-propagation, in general).
However, quoting the TORCH.SPARSE 2 documentation:

This API is currently experimental and may change in the near future.

So maybe this is just a (hopefully known) limitation of a work
in progress.
Best.
K. Frank