KeyError: 'labels' in   training_step in transformers/trainer.py
## Environment info
- `transformers` version: 3.5.1
- Platform:
- Python version: 2.7 
- PyTorch version (GPU?): GPU 
- Tensorflow version (GPU?): - 
- Using GPU in script?: - 
- Using distributed or parallel set-up in script?: - 

### Who can help
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 T5: @patrickvonplaten
 examples/seq2seq: @patil-suraj

## Information
I am using finetune_seq2seq model, the issue arises inside  training_step in trainer function, and it gives error that there is not "labels" inside the batch, below please find the stack trace, I spent the whole day I could not get why dataloader does not return back the labels, I would be grateful to point me to some possible reasons why this behaviour might happen and assisting me figuring this out. thanks 

```
Traceback (most recent call last):
  File "finetune_t5_trainer.py", line 250, in <module>
    main()
  File "finetune_t5_trainer.py", line 183, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/trainers/trainer.py", line 784, in train
    tr_loss += self.training_step(model, inputs)
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/trainers/trainer.py", line 1125, in training_step
    loss = self.compute_loss(model, inputs)
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/trainers/t5_trainer.py", line 338, in compute_loss
    labels = inputs.pop("labels")
KeyError: 'labels'

```

This happens in fact after the first epoch, could you think of the reason why this is the case? I tested the dataloader alone and it generates the epochs properly for any number of epochs.<|||||>during the first epoch my batches have all info
`batch inside multitask  dict_keys(['input_ids', 'attention_mask', 'decoder_input_ids', 'labels'])`

After first epoch, they miss the labels, my dataloader has an inner dataloader and I checked this is not called anymore after epoch 1.

`batch inside multitask  dict_keys(['input_ids', 'attention_mask', 'decoder_input_ids'])`

thanks a lot in advance, I am really struggling with this issue, appreciate any helps/thoughts on this. 
<|||||>here is the structure of multi-task dataloader, which is my train_dataloader,  could you point me what might happen after first epoch? could you point me to any changes you introduce to the train dataloader after epoch 1? thanks 

```
class TaskDataLoader:
    """Wrapper around dataloader to keep the task names."""
    def __init__(self, task_name, dataset, batch_size=8,
                 collate_fn=None, drop_last=False, num_workers=0, sampler=None):
        self.dataset = dataset
        self.task_name = task_name
        self.data_loader = DataLoader(self.dataset,
                                      batch_size=batch_size,
                                      sampler=sampler,
                                      collate_fn=collate_fn,
                                      drop_last=drop_last,
                                      num_workers=num_workers)
    def __len__(self):
        return len(self.data_loader)

    def __iter__(self):
        for batch in self.data_loader:
            print("### batch inside taskdataloader ", batch.keys())
            yield batch



class MultiTaskDataLoader:
    """Given a dictionary of task: dataset, returns a multi-task dataloader
    which uses temperature sampling to sample different datasets."""

    def __init__(self,  tasks_to_datasets, batch_size=8, collate_fn=None,
                 drop_last=False, num_workers=0, temperature=100.0):
        # Computes a mapping from task to dataloaders.
        self.task_to_dataloaders = {}
        for task, dataset in tasks_to_datasets.items():
            dataloader = TaskDataLoader(task, dataset, batch_size,
                collate_fn=collate_fn, drop_last=drop_last, num_workers=num_workers)
            self.task_to_dataloaders.update({task: dataloader})
        self.tasknames = list(self.task_to_dataloaders.keys())

        # Computes the temperature sampling weights.
        self.sampling_weights = self.temperature_sampling(self.dataloader_sizes.values(), temperature)
        self.dataiters = {k: cycle(v) for k, v in self.task_to_dataloaders.items()}

    def temperature_sampling(self, dataset_sizes, temp):
        total_size = sum(dataset_sizes)
        weights = np.array([(size / total_size) ** (1.0 / temp) for size in dataset_sizes])
        return weights/np.sum(weights)

    @property
    def dataloader_sizes(self):
        if not hasattr(self, '_dataloader_sizes'):
            self._dataloader_sizes = {k: len(v) for k, v in self.task_to_dataloaders.items()}
        return self._dataloader_sizes

    def __len__(self):
        return sum(v for k, v in self.dataloader_sizes.items())

    def num_examples(self):
        return sum(len(dataloader.dataset) for dataloader in self.task_to_dataloaders.values())

    def __iter__(self):
        outputs = {}
        for i in range(len(self)):
            taskname = np.random.choice(self.tasknames, p=self.sampling_weights)
            dataiter = self.dataiters[taskname]
            #outputs["batch"] = next(dataiter)
            #outputs["task"] = taskname
            #outputs = next(dataiter)
            #outputs["task"] = taskname 
            outputs = next(dataiter)
            print("### batch inside multitask ", outputs.keys())
            yield outputs

# Example how this dataloader works.
if __name__ == "__main__":
    batch_size = 10
    num_shards = 2
    rank = 0
    dataset1 = load_dataset('glue', 'rte', split="train[:16]")
    dataset2 = load_dataset('glue', 'cola', split="train[:32]")
    trainer = Trainer({'dataset1': dataset1, 'dataset2': dataset2})
    dataloader = trainer.get_train_dataloader()
    print("### length ", len(dataloader))
    for epoch in range(1000):
      for i, batch in enumerate(dataloader): #islice(dataloader, 5):
        print("## epoch ", epoch, " i ",  i) #batch) #batch)


```
<|||||>solved in #8859