Ya seems reasonable. Do you have a rough sense of the relative loading time overhead in all 3 methods?

Loading batch by batch
Loading groups of batches
Loading entire dataset at once (if it hypothetically did fit in memory)

I guess I’m thinking in terms of X times slowdown for 1 and 2 versus 3. Maybe just some rough guesses would be nice.