Special Tokens Not Working as Expected in Bert Tokenizer
## Environment info
- `transformers` version: 4.18.0
- Platform: macOS-12.0.1-arm64-arm-64bit
- Python version: 3.8.13
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
Tokenizers: @SaulLu

## Information
Using the BERT tokenizer and wanted to add my own special tokens, but am not getting the expected behavior

## To reproduce

```
    tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
    n_added_tokens = tokenizer.add_tokens(["<CONCEPT>", "</CONCEPT>"], special_tokens=True)
    print(n_added_tokens)
    context = "Hi I am taking <CONCEPT> Advil </CONCEPT>"
    context_tokens = tokenizer(context).input_ids
    print(tokenizer.convert_ids_to_tokens(context_tokens))
    context = "<CONCEPT>"
    context_tokens = tokenizer(context).input_ids
    print(tokenizer.convert_ids_to_tokens(context_tokens))
```

**Output:**
```
2
['[CLS]', 'hi', 'i', 'am', 'taking', '<', 'concept', '>', 'ad', '##vil', '<', '/', 'concept', '>', '[SEP]']
['[CLS]', '<CONCEPT>', '[SEP]']
```

## Expected behavior
**Expected Output:**
```
2
['[CLS]', 'hi', 'i', 'am', 'taking', '<CONCEPT>', 'ad', '##vil', '</CONCEPT>', '[SEP]']
['[CLS]', '<CONCEPT>', '[SEP]']
```

Seems like when we pass the special token alone it somehow works, but not in more text . 
Thank you very much for sharing this issue!

It looks like you've hit on a problem in the data workflow in our slow tokenizers (which has been around for a very long time apparently :scream_cat: )!


To solve your problem, I suggest you do instead:
```python
from transformers import BertTokenizer

model_name = "bert-base-uncased"
tokenizer_s = BertTokenizer.from_pretrained(model_name)

n_added_tokens = tokenizer_s.add_special_tokens({"additional_special_tokens":["<CONCEPT>", "</CONCEPT>"]})
print(n_added_tokens)
context = "Hi I am taking <CONCEPT> Advil </CONCEPT>"
context_tokens = tokenizer_s(context).input_ids
print(tokenizer_s.convert_ids_to_tokens(context_tokens))
# ['[CLS]', 'hi', 'i', 'am', 'taking', '<CONCEPT>', 'ad', '##vil', '</CONCEPT>', '[SEP]']
```

_I'll take this opportunity to share my analysis on the origin of the error (**you can ignore this section if you're not interested, I prefer to keep track of it for the futur me)**:_
This behavior problem is due to the fact that being a special token for a slow tokenizer means at least 2 things:
1. the token must be listed in one of the `SPECIAL_TOKENS_ATTRIBUTES` keys 
https://github.com/huggingface/transformers/blob/ee209d4d016e2ef1b2e73c4be64ad43895bc7e27/src/transformers/tokenization_utils_base.py#L775-L784
2. the token must be matched before the ("simplified") normalization step if it exists, i.e. the lowercase pass. 
https://github.com/huggingface/transformers/blob/ee209d4d016e2ef1b2e73c4be64ad43895bc7e27/src/transformers/tokenization_utils.py#L448-L455
What happens here is that the `add_tokens` method has no impact on feature 1 - which makes sense because we wouldn't know what kind of token special it is. What is annoying, however, is that since this token is never associated with any of the special token attributes it is not listed in `self.all_special_tokens` and so in the `_create_trie` method its occurrence cannot be noticed.

I would need to think a bit more about how to avoid other users having the same problem as you (without breaking the rest) :smile: 
<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.