providing an example with a dummy iterative dataloaders
Hi
I have tested the trainer.py with iterative datasets and this does not work in distributed case, I shard the data across the cores. Could you please assist me in providing me with a dummy iterative dataloader for finetune_seq2seq.py model which runs fine with xla_spawn.py on TPU so I get some understanding which functions needs to be implemented. I really need to make this work, and trainer.py does not seem to work with iterative datasets or I am missing how to do it properly. thanks 

@sgugger 
Trainer does indeed not work in distributed fashion with iterative datasets. You need to convert your iterative dataset to a regular dataset for the time being.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.