I just load a list of files in the __init__. I don’t think it is process spawning that’s causing the bottleneck. The call to __getitem__ has to do a lot of IO so I think when the dataloader starts and none of the workers have prefetched any data it takes a while to start. In the middle of an epoch this isn’t a problem because some of the worker have already fetched data.