Oh, it seems that torch.spmm is confusing.
With torch.sparse.mm, you can have gradients all right.
a = torch.randn(2, 3).to_sparse().requires_grad_(True)
b = torch.randn(2, 3, requires_grad=True)
torch.sparse.mm(a,b.t()).sum().backward()

Best regards
Thomas