Hi Frank. You summarized the problem perfectly, thanks a lot for your time!!
Just to clarify:

Indeed I’m basically finding the hidden vector x that minimizes the residual |y - Ax|

I do have set requires_grad = False for the matrix A
I assume torch knows that the transpose of a sparse matrix is sparse too, otherwise I would have had memory issues (did I mention that my matrix is huge?  )
To give some context: my matrix A is roughly 850.000 x 210.000 with about 0.1% of non-zero coefficients. For this reason, I don’t think that a full matrix A is ever computed, otherwise my RAM will simply be too small to hold it.
I forgot to remove RF.mask, sorry for that. That is a matrix of one and zeros that allows me to weight the MSE loss in order to avoid some of the data. The situation doesn’t change if I remove it (its requires_grad is set to False)