Is memro_dataset loading the data in its __init__ or does it already contain all the data?
Each worker would create a copy of this dataset, which would explain the slight increase in 1, 2, 3.
However, each process in the DDP setup would create the specified number of workers, so that you would end up with more workers in the end. I don’t know, why the creation takes so much more time though.