A slow tokenizer cannot add a token with the argument `lstrip=False`
## Environment info
```
- `transformers` version: 4.17.0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (False)
- Tensorflow version (GPU?): 2.8.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
```

## Information

When adding a token to a tokenizer with a slow backend or to a tokenizer with a fast backend, if you use the `AddedToken` class with the `lstrip=False` argument, the final tokenization will not be the same.

My impression is that the slow tokenizer always strips and that the fast tokenizer takes this argument into account (to be confirmed by the conclusion of [this issue](https://github.com/huggingface/tokenizers/issues/959) in the tokenizers library).

## To reproduce

```python
from transformers import AutoTokenizer, AddedToken

def print_tokenizer_result(text, tokenizer):
    ids = tokenizer.encode(text, add_special_tokens=False)
    tokens = tokenizer.convert_ids_to_tokens(ids)
    print(f"tokens: {tokens}")

model_name = "patrickvonplaten/norwegian-roberta-base"

tokenizer_init = AutoTokenizer.from_pretrained(model_name)
tokenizer_init.save_pretrained("local_tokenizer")

model_name = "local_tokenizer"
tokenizer_s = AutoTokenizer.from_pretrained(model_name, use_fast=False)
tokenizer_f = AutoTokenizer.from_pretrained(model_name, use_fast=True)

new_token = "added_token_lstrip_false"
tokenizer_s.add_tokens(AddedToken(new_token, lstrip=False))
tokenizer_f.add_tokens(AddedToken(new_token, lstrip=False))

text = "Example with added_token_lstrip_false"
print("Output for the fast:")
print_tokenizer_result(text, tokenizer_f)
print("\nOutput for the slow:")
print_tokenizer_result(text, tokenizer_s)
```

Output:
```
Output for the fast:
tokens: ['Ex', 'amp', 'le', 'Ġwith', 'Ġ', 'added_token_lstrip_false']

Output for the slow:
tokens: ['Ex', 'amp', 'le', 'Ġwith', 'added_token_lstrip_false']
```

## Expected behavior

Output:
```
Output for the fast:
tokens: ['Ex', 'amp', 'le', 'Ġwith', 'Ġ', 'added_token_lstrip_false']

Output for the slow:
tokens: ['Ex', 'amp', 'le', 'Ġwith', 'Ġ', 'added_token_lstrip_false']
```

The logic for `lstrip` **is** implemented for slow tokenizers actually:

https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils.py#L519

The main thing I think is that in that function,the added token is NOT in the `all_special_added_tokens_extended`. If it was, it would work.

Now the function `add_tokens` starts by removing all information about the `AddedToken` which if it didn't maybe we could keep parity here. https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils.py#L411

So my conclusion is that the core issue is that the slow tokenizer discards all the info you are sending it, it probably shouldn't.
That being said, the simplest backward compatible fix, might not be obvious.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.