it looks like when num_workers>batch_size, the same data will be fed to the neural net. I was using num_worker = 8 and batch_size = 1, and the same example is fed as input for multiple times. I think it’s a bug or something.