The source code for the DataLoader logic is too complex for me, but I see two reasons why #2 would be the approach used in practice:


@apaszke said it,
creating a batch from the two workers would require waiting for them to finish their task (getting data) before returning the batch, which is inefficient and (I guess depending on the architecture, data structure, storage etc…) useless compared to a single process loading sample1, sample2, sample3 and sample4 and then returning the batch.

What is your use case?