Hi @ptrblck, I did some test for dataloader and pipeline, the test looks like this

Experiment 1: With preload data, data pre load into memory

replay_mem = {}
for fi, batch in enumerate(my_data_loader):
      replay_mem[fi] = batch

# Training with all data in memory

for i in range(0,epoch)
    for fi, batch in enumerate(replay_memory.items()):
     Train(batch)


Experiment 2: Without preload data, the data is loaded via dataloader

for i in range(0,epoch)
    for fi, batch in enumerate(ny_data_loader):
     Train(batch)

The experiment 1 has less time, and time diff compared with experiment 2 is about the data loading time(if I just test the data loading). I imagine if we have pipeline that training just consumes multi worker data loader, if training time is higher than multi worker data loader, so data loader time will be completely shaded? (data loading is always faster and thus training always have data)?