Hi there, since so many people clicked there, i just wanted to say that i simplified a bit the code, and that, in fact, the easiest for people who train networks with memory could be to just use an IterableDataset where you yield the worker’s id at the same time as your data. Otherwise, I did a wrapper around Pytorch DataLoader that just concatenates the data from different FIFOs indexed by the worker’s id.