Tokenizer encoding skips � character
## Environment info
- `transformers` version: 4.5.1
- Platform: Linux-4.4.0-19041-Microsoft-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- PyTorch version (GPU?): 1.8.1+cu102 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

- tokenizers: @LysandreJik

## Information

Model I am using (Bert, XLNet ...): Electra

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("google/electra-small-discriminator")
c = "foo � bar"
print(f"c[4:5]={c[4:5]}")
e = tokenizer(c, return_offsets_mapping=True)
print(repr(e))
"""
{'input_ids': [101, 29379, 3347, 102], 'token_type_ids': [0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1], 'offset_mapping': [(0, 0), (0, 3), (6, 9), (0, 0)]}
"""
i = e.char_to_token(4)
print(f"i={repr(i)}")  # i=None
```

## Expected behavior

Problem: � character was not encoded by the tokenizer.

� character should be encoded as some token <UNK> or otherwise.

Said character appears in the SquadV2 dataset with ID `5acd29f507355d001abf3774`:
```
Question
What is the glyph that Apple's Last Resort font displays?

Context
Rendering software which cannot process a Unicode character appropriately often displays it as an open rectangle, or the Unicode "replacement character" (U+FFFD, �), to indicate the position of the unrecognized character. Some systems have made attempts to provide more information about such characters. The Apple's Last Resort font will display a substitute glyph indicating the Unicode range of the character, and the SIL International's Unicode Fallback font will display a box showing the hexadecimal scalar value of the character.

Answer
�
```

Maybe @n1t0 has an idea!<|||||>This is totally expected behavior. This tokenizer uses the same cleanup steps that were used in BERT, and this character is specifically removed.

Cf here on line 492:
https://github.com/huggingface/transformers/blob/32dbb2d/src/transformers/models/bert/tokenization_bert.py#L487-L498