It is about pytorch. I thought Ignine has some elegant solution for this behavior. As you suggested add callback @trainer.on(Events.EPOCH_STARTED)
class DatasetRandom(Dataset):    
    def __len__(self):
        return 2
    def __getitem__(self, idx):
        return (np.random.randint(1000, size=1), idx%2)

train_loader = DataLoader(DatasetRandom(), batch_size=1, num_workers=2)

for e in range(2): 
    print('epoch',e)
    for i, (images, labels) in enumerate(train_loader):
        print(i, images, labels)

epoch 0
0 tensor([[492]]) tensor([0])
1 tensor([[492]]) tensor([1])
epoch 1
0 tensor([[492]]) tensor([0])
1 tensor([[492]]) tensor([1])

Ignite: 0.4.1, PyTorch: 1.4.0