I assume the data fits completely into memory?
If so, you could load the .mat file using scipy in your Dataset's __init__ method, get a single sample in __getitem__ by indexing and returning the tensor via torch.from_numpy.