are you looking for matrix operations over matrices that are sparse by block ? That would surely reduce you dimensionality problem.
In that case, what about using several Linear layers that are not sparse instead of a large sparseLinear layer?