I’m not sure how to implement your data loading logic using sparse, but for your first suggestion you could create a custom Dataset and load the samples lazily.
Using a DataLoader the batches can be preloaded while your GPU is busy with the training step.
Have a look at the data loading tutorial 21 to create your Dataset.