I want to multiply two dense matrices A(N, d) and B(d, N). The resulting matrix will be of size N x N, where N is very large and I can’t use normal matrix multiplication function to do this.
However, I only want values at a few positions which are specified in a sparse matrix C(N, N). For example, I only want E non-zero entries in C.
My problem is the same as discussed in Matrix multiplication with values only at masked entries 1.
However, I don’t find a memory efficient way to solve this problem.
The solution provided by tom take a memory complexity of O(Ed).
Spliting it into chunks does not help as it still occupies memory for gradient ackpropagation.