Something like this:
import torch

x = torch.rand(5,6)
# Sparse matrix of (0, 1) = 1; (2, 1) = 2, (3, 4) = 3
sparse = torch.sparse.FloatTensor(
    torch.LongTensor([[0, 2, 3], [1, 1, 4]]), # Indicies
    torch.FloatTensor([1, 2, 3])) # values

print(x)
print(sparse.to_dense())
print(torch.mm(sparse, x))
# This won't actually save space or compute, since it's so dense,
# but it will be a sparse tensor representation.
print(torch.smm(sparse, x))

Simply construct a list of your sparse tensors, and loop over them to do the batch mm.