Solved by vincentqb in post #2 
              
              
                One way to manage async iterators is to use a background iterator to prefetch ahead of time. But the exact setup depends on what you are trying to do of course  
If you start with lots the small files (for a total of 1 GB), then you could create a dataset that reads them on __getitem__. You could ei…