Make ViltModel forward method arguments (inputs_embeds and image_embeds) consistent
### Feature request

The inputs_embeds arguemnt in Vilt expects the word embeddings (which is [cls_token_emb, token_embs, sep_token_emb]) of the input without adding the pos or token_type embs yet, while the image_embeds argument expects the processed embeddings [cls_token_emb, patch_embs] + pos_embs + token_types , which seems a bit inconsistent?  

### Motivation

I was a bit confused why the following code was not producing the same output and then read the source code an realized the issue:

```
from transformers import ViltProcessor, ViltModel
from PIL import Image
import requests

# prepare image and text
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
text = "hello world"

processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-mlm")
model = ViltModel.from_pretrained("dandelin/vilt-b32-mlm")

inputs = processor(image, text, return_tensors="pt")
outputs_1 = model(**inputs)


# using embeds (the wrong way)
img_embs, img_mask, (patch_index, (h, w)) = model.embeddings.visual_embed(inputs['pixel_values'], inputs['pixel_mask'])
txt_embs = model.embeddings.text_embeddings(inputs['input_ids'])
outputs_2 = model(inputs_embeds=txt_embs, image_embeds=img_embs, pixel_mask=img_mask)


# seems to be the correct way
img_embs, img_mask, (patch_index, (h, w)) = model.embeddings.visual_embed(inputs['pixel_values'], inputs['pixel_mask'])
txt_embs = txt_emb = model.embeddings.text_embeddings.word_embeddings(inputs['input_ids']) # word_embeddings instead
outputs_3 = model(inputs_embeds=txt_embs, image_embeds=img_embs, pixel_mask=img_mask)


```

outputs_2 and outputs_3 are (almost) identical (they aren't exact matches but the total difference between all entries is around -0.009) while outputs_1 and outputs_2 are just different.

### Your contribution

Make both arguments either take the processed embeddings (after adding everything) or just the raw embeddings before adding anything. unless ofc, this is a deliberate decision made for reasons I don't know :o. If it is not intended, then I can try and submit a pr that fixes it.
Hi,

Thanks for reporting. I'm afraid we can't change it for backwards compatibility reasons, as people might have already used `image_embeds` with the way it is implemented now. Cc'ing @sgugger to confirm<|||||>Indeed. The documentation can be clarified however, to make this less surprising to users.<|||||>makes sense! i guess the docs could be indeed slightly clarified <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.