and 2.: since DataLoaders use multi-processing the Datasets will be copied and each worker will work in its own Dataset instance, so there shouldn’t be any read/write conflict. However, this would also mean that your memory usage should increase with the used number of workers.


You could compare both approaches, but note that the “lazy loading” will most likely not work using multiple workers, as the data would be stored on a replica and you would need to use shared arrays/dicts as given in this example 15. Also to avoid reloading the complete data, you could pre-load it outside of the Dataset.__init__ and just pass it as an argument. Otherwise the dataset will be recreated in each epoch.