After using transformers.Trainer, batch data in my dataset become empty
### System Info

- `transformers` version: 4.22.1
- Platform: Linux-3.10.0-1127.el7.x86_64-x86_64-with-glibc2.27
- Python version: 3.9.7
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1.First I define my own type of dataset
```
class Seq2SeqDataset(data.Dataset):
    def __init__(self, data_dir):
        self.datas = read_json(data_dir)

    def __len__(self):
        return len(self.datas)

    def __getitem__(self, index):
        return self.datas[index]
```
2.then I load the training data
`trainset = Seq2SeqDataset(args.train_data_dir)`
I make sure I successfully load the dataset
by print the len and try to call item within the dataset, all work fine
3.then I define my own collator such as the following one
```
class DataCollatorForSeq2Seq:
    def __init__(self, tokenizer, padding: bool = True, max_length: int = 512):
        self.tokenizer = tokenizer
        #self.model = model
        self.padding = padding
        self.max_length = max_length

    def __call__(self, batch):
        features = self.collator_fn(batch)
        return features

    def collator_fn(self, batch):
        print(batch)
        results = map(preprocess, batch)
        inputs, targets, _ = zip(*results)

        input_tensor = self.tokenizer(inputs,
                                      truncation=True,
                                      padding=True,
                                      max_length=self.max_length,
                                      return_tensors="pt",
                                      )
................
```
4.
this collator is used in the Trainer

```
collator = DataCollatorForSeq2Seq(T5_tokenizer, max_length=args.max_seq_len)
trainer = Trainer(
        tokenizer=T5_tokenizer,
        model=T5_model,
        args=training_args,
        data_collator=collator,
        train_dataset=trainset,
        eval_dataset=devset
    )
```
5.the strange things is that all the data in batch become empty, each item of the origin dataset is a dict
6.the trainer argument is defined as follow
```
training_args = TrainingArguments(
            num_train_epochs=args.epochs,
            per_device_train_batch_size=args.batch_size, 
            per_device_eval_batch_size=args.batch_size,
            logging_steps=args.logging_steps,
            weight_decay=args.weight_decay,
            evaluation_strategy=args.evaluation_strategy,
            eval_steps=args.eval_steps,
            load_best_model_at_end=True,
            learning_rate=args.learning_rate,
            warmup_steps=args.warmup_steps,
            warmup_ratio=args.warmup_ratio,
            output_dir=args.output_dir,
            save_total_limit=args.save_total_limit,
            lr_scheduler_type=args.lr_scheduler_type,
            gradient_accumulation_steps=args.gradient_accumulation_steps,
            dataloader_num_workers=args.dataloader_num_workers)
```

### Expected behavior

the batch data suppose to be the batch from our dataset, however it is all empty
It's very hard to help without knowing what your dataset is. The `Trainer` will drop columns in your dataset that are not accepted by your model, so it might be that. Use `remove_unused_columns=False` in your `TrainingArguments` and see if it changes anything.<|||||>> It's very hard to help without knowing what your dataset is. The `Trainer` will drop columns in your dataset that are not accepted by your model, so it might be that. Use `remove_unused_columns=False` in your `TrainingArguments` and see if it changes anything.

Thanks for the hint. I change the dict into tuple, in the dataset and it works fine now. It's not ideal though. Not sure whether if the trainer won't support the dict format in the torch Dataset<|||||>> 
```

class Seq2SeqDataset(data.Dataset):
    def __init__(self, data_dir):
        self.datas = read_json(data_dir)

    def __len__(self):
        return len(self.datas)

    def __getitem__(self, index):
        return self.datas[index]['passage'], self.datas[index]['question'], self.datas[index]['answer'], self.datas[index]['answer_start']

```
That' s how I modified my dataset now, which will work