Solved by lmnt in post #2 
              
              
                Seems it is just not currently possible to do dense X sparse > dense (please correct me if wrong) 
Workaround is to just have sparsity manually. I.e. make the “sparse” matrix dense, and multiplying by a mask of 1s and 0s each trial. Main goal here was to have a dense input layer mm by a sparse weigh…