VisualBert ValueError: visual_embeds can not be of class 'NoneType' when running on text only
## Environment info

- `transformers` version: 4.8.2
- Platform: Linux-4.15.0-143-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help

@sgugger @gchhablani 

## Information

Model I am using (Bert, XLNet ...): VisualBERT

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoModel, AutoTokenizer

model_name_or_path = 'uclanlp/visualbert-vqa-coco-pre'
tokenizer_name_or_path = 'bert-base-uncased'

model = AutoModel.from_pretrained(model_name_or_path,
                                  cache_dir='cache')
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name_or_path,
                                          cache_dir='cache')
inputs = tokenizer('This is a test.', return_tensors='pt')

encoder_out = model(**inputs)
```
Gives error:
```python
Traceback (most recent call last):
  File "/cw/liir/NoCsBack/testliir/rubenc/miniconda3/envs/tsenv/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-0aa46003b81a>", line 12, in <module>
    encoder_out = model(**inputs)
  File "/cw/liir/NoCsBack/testliir/rubenc/miniconda3/envs/tsenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/cw/liir/NoCsBack/testliir/rubenc/miniconda3/envs/tsenv/lib/python3.8/site-packages/transformers/models/visual_bert/modeling_visual_bert.py", line 777, in forward
    raise ValueError(
ValueError: `visual_embeds` can not be of type <class 'NoneType'> when using a VisualBert Model.
```

## Expected behavior

I would like to encode only text, not image_features. The [docs](https://huggingface.co/transformers/model_doc/visual_bert.html#transformers.VisualBertModel.forward) for VisualBert say that the `visual_embeds` parameter is optional. The forward method of `VisualBertEmbeddings` seems to work when 
`visual_embeds` is `None`, so I think the only thing that prevents encoding text only seems to be the check in the forward method of `VisualBertModel`? Or am I missing something? 🙂 

Hi @rubencart

I think this is an issue with the documentation 😅 I can fix that.

Can you share your use case where you only want to pass textual inputs to VisualBERT? 

I placed this check only to prevent usage of model without any visual embeddings.

CC @patil-suraj <|||||>I want to encode text, to later use it as input for a visual downstream task. Instead of using an encoder that has been pretrained on text only, it made sense to me to try to encode it with an encoder whose pretraining was more visually informed.

Can I not use VisualBert for this? Technically, if you just remove the check, wouldn't this work? :-)<|||||>@rubencart I think you should be able to. 
Wdyt @patil-suraj, should this be allowed?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Yes, this should be allowed, feel free to open a PR if you want @gchhablani :)