Collator only gets keys from the dataset which are inputs to the model
### System Info

- `transformers` version: 4.21.3
- Platform: macOS-12.1-arm64-arm-64bit
- Python version: 3.9.10
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.10.2 (False)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:
- Use a dataset with certain keys expected by a custom collator but not the model inputs
- Pass the dataset and custom collator to the huggingface trainer
- Run training and the collator will not be passed the correct keys

### Expected behavior

Huggingface Trainer automatically removes keys from the dataset which aren't needed by the model, but doesn't allow for the possibility that the collator might take different inputs than the model. This is unexpected as the collator is passed the data prior to the model so when stripping unused keys it should be done after the collator not prior.
You can just the option `remove_unused_keys=False` from your training arguments in this case.<|||||>Okay, thanks! I looked through before and didn't see anything, actually CTRL-F'd `key` but there was nothing. I believe it has actually been renamed `remove_unused_columns` which is why I didn't find it. Thanks for the help again! <|||||>Ah yes, sorry about the wrong name!<|||||>No worries, thanks for the help.