_iterable:

My dataset is too huge to hold the entire data in the buffer, so I would like to hold only intermediate batches returned by the dataloader in the buffer.


Would you like to create a caching mechanism for e.g. 10% of the data and reuse it in the next epoch? If you are filling (and clearing) the cache with new samples in each iteration, I would expect to see mostly misses, since your would basically implement a running window which doesn’t contain the next sample.



 _iterable:

If I have static buffer, can I provide a custom collate function to the datalaoder, which writes the batch into a buffer directly?


I think that’s possible, since my example does the same inside the Dataset.__getitem__.



 _iterable:

If so, do I need to do special handling for cases where num_workers > 0?


Yes, I also think you would have to reuse my approach of using the shared array.