Seems it is just not currently possible to do dense X sparse > dense (please correct me if wrong)
Workaround is to just have sparsity manually. I.e. make the “sparse” matrix dense, and multiplying by a mask of 1s and 0s each trial. Main goal here was to have a dense input layer mm by a sparse weight matrix to produce a dense next layer. Then of course to apply backprop to these sparse layers, and needed the sparse values to not be updated. By multiplying by the mask, those unwanted changes resets values to 0.