German Bert tokenizer does not recognize (some) special characters (!,?,...)
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

**Bert**

Language I am using the model on (English, Chinese ...):

**German**

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

`tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')`
`tokenizer.encode("Hallo!", add_special_tokens=False)`

returns:

`[5850, 26910, 2]`

where `2` is the index of '[UNK]' (unknown) token. Same would happen with 'Hallo?'. In the vocab.txt we have tokens '##!' and '##?' but not '?' or '!'. On the other hand some special characters are recognized, like ':' or ';'.

## Expected behavior

At least such common characters as '!' and '?' should be recognized by tokenizer


## Environment

* OS: Ubuntu
* Python version: 3.7
* `transformers` version (or branch): 2.3
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Sounds like this was fixed by @Timoeller in #3618, @andrey999333 <|||||>Thanks for referencing. This bug should be fixed with the changes we applied to the vocabulary.
Find more infos in the separate issue here: deepset-ai/FARM/issues/60