When running the code above, data do not get distributed as expected. That is, if I train a model in the same circumstance as in the example (data length : 1000, batch size : 100, num gpus : 2), each process runs 10 iterations per epoch, not 5. Why would DistibutedSampler behave differently?

What is the value self.world_size in the make_dataloader function?
If it is 1 each process will run 10 iterations.