Each worker loads a complete batch, i.e. batch_size samples.
The used indices are specified by the sampler or if no sampler is defined by the shuffle option in the DataLoader, which will internally create a SequentialSampler or a RandomSampler.
How did you profile the code and what does



 t.ouyang:

BUT when using batch_size=1 why the loading speed still follows num_workers=4 > num_workers=2 > num_workers=0 ?


mean?