One of the ways I heard good things about was to have one huge tensor in a file and memmap it. Then you would want to solve indexing.
Another perhaps even more obvious choice could be splitting the ds, loading the right oart if necessary and adapt the dataloader’s random sampler to draw from block-form permutations.
I look forward to hearing about your experience.
Best regards
Thomas