Issues with Training VisionEncoderDecoder with Seq2SeqTrainer
Hello Team, 

I am trying to code up a VisionEncoderDecoder with VIT + BERT and finetune it with `Seq2SeqTrainer.` (Presuming the support exists).

```python
from transformers import VisionEncoderDecoderModel
from transformers import ViTFeatureExtractor, AutoTokenizer
import torch 

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
feature_extractor = ViTFeatureExtractor.from_pretrained('google/vit-base-patch16-224-in21k')
vitbert = VisionEncoderDecoderModel.from_encoder_decoder_pretrained('google/vit-base-patch16-224-in21k', 'bert-base-uncased')
vitbert.to(device)
print("Model loaded")
```
VITModel accepts `pixel_values`  but the VisionEncoderDecoder model apparently fails to recognize the preprocessed inputs with image encodings as `pixel_values` and `labels`. So I changed the `pixel_values` into `input_ids` for the model to accept? (not sure if I am in the right direction) .

**The Pre-Process snippet** 
```python

input_encodings = feature_extractor(images=inputs, return_tensors="pt")
input_encodings["input_ids"] = input_encodings["pixel_values"]
```

**Feature casting snippet**
```python
from datasets import Features, Array3D, Sequence, Value
features = Features({
  "input_ids": Array3D(dtype="float32", shape=(3, 224, 224)),
  "labels": Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None)
})
preprocessed_train_ds = tokenized_train_datasets.map(ds_preprocess_function, batched=True, features=features)
preprocessed_val_ds = tokenized_eval_datasets.map(ds_preprocess_function, batched=True, features=features)
```

But `Trainer.train` throws the below error.

```python
ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided ['labels']
```

Not sure what I am missing here. 

Please advice.
Hi,

Did you take a look at my [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_Seq2SeqTrainer.ipynb)?

I just tested, everything seems to work fine on my end. You don't need to rename the `pixel_values` to `input_ids`.

<|||||>> Hi,
> 
> 
> 
> Did you take a look at my [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_Seq2SeqTrainer.ipynb)?
> 
> 
> 
> I just tested, everything seems to work fine on my end. You don't need to rename the `pixel_values` to `input_ids`.
> 
> 
> 
> 

Thanks a lot will look at. <|||||>All good now ! 

I supplied the decoder tokeniser instead of the encoder feature extractor  in the tokeniser parameter while setting up the seq2seq trainer. 

Thanks 