If you set num_workers > 0 in your DataLoader, multiple workers will be created and load your batches of data in the background.
E.g. for multiple workers you will get batch1 from worker1, batch2 from worker2 etc., then it will start with worker1 again.
I’m not sure, if the order is somehow enforced or if the next ready worker just puts his data into the queue.
In the latter case, it might be that somehow the order might be broken, and worker2 finishes before worker1.
This would yield your data like:
a1, b1
a2, b2,
a4, b4,
a3, b3,
...

Would this be troublesome or can you handle this case?