claudiacorreia60:

Do the workers fetch batches for the next epoch before it starts, or the batches of an epoch only start being fetched when the epoch starts?


It only start with the epoch, when the iterator is created. Meaning, when you do:
for sample in dataloader:

You can specify a sampler (doc 27) when you create the Dataloader that is reponsible to draw the samples.
You can control the sampler to force a certain pattern in the content of the batch (and thus what the workers will load).