Hi,
You might want to try out the new IterableDataset 124 on PyTorch master or nightly release. However note that to correctly use num_workers>0 you will have to configure your dataset based on the worker info 54 to avoid generating duplicate data.