The DataLoader is prefetching samples, which can be set by prefetch_factor as described in the docs.
The default value is set to 2 * num_workers so 4 in your case, which explains the loading of 4 samples at the beginning.