It might depends on your sparsity factor and the size of your matrices, for my use-case scipy was better.
I’d say, the easiest is definitely to use scipy to start with. Converting dense Tensors to numpy arrays is for free, so depending on how you store your data you could do the torch -> numpy sparse conversion very efficiently.
Then, if you find it’s a bottleneck, I’d look for other alternatives, but my experience with solving linear systems with cusolver was not the best.