I think the best way would be  to use torch.save and torch.load.
But as IO is extremely slow I think you should avoid loading batches 1 per 1.
Even if your entire dataset cannot fit into memory, I’m sure you can fit more than only one batch.
Here is how I do it in my Dataset class
On my disk I have saved several files containing my training examples, each file contains 250 000 examples and it’s just a single tensor of dims (250000,firstDim,secondDim…) saved using torch.save.
My Dataset loads only one of these files using torch.load, and when it is asked for a batch, I simply select the batch from my giant tensor using tensor.narrow and I clone the result into GPU memory (to ensure minimal GPU usage, only one batch is loaded into GPU). When the dataloader run out of batches to select from the giant tensor, it simply load it again from a different file.
I found that this method is really efficient because I/O is minimal, and you can really tune it depending on how much RAM you want to use.