single_word option of new tokens are disabled by save_pretrained when we save and reload a tokenizer twice
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

I set up environment as follows:

```bash
conda create -n test python=3.9
conda activate test

pip install transformers
# I got transformers 4.10.2 and tokenizers 0.10.3
```

Other details:

- `transformers` version: 4.10.2
- Platform: Linux-5.11.0-27-generic-x86_64-with-glibc2.31
- Python version: 3.9.6
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help

* tokenizer: @LysandreJik

## Information

`single_word` option of new tokens are disabled by `save_pretrained` when we save and reload a tokenizer twice.

## To reproduce


```python
from transformers import AutoTokenizer
from tokenizers import AddedToken

# Load tokenizer and add tokens.
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
new_vocab = [AddedToken("some_word", single_word=True), AddedToken("some_words", single_word=True)]
tokenizer.add_tokens(new_vocab)

def check_tokenizer():
    print(tokenizer.convert_ids_to_tokens(tokenizer.encode("some_words", add_special_tokens=False)))

check_tokenizer()

# Save and reload tokenizer
tokenizer.save_pretrained("first_save")

tokenizer = AutoTokenizer.from_pretrained("./first_save")

check_tokenizer()

# Save and reload tokenizer again
tokenizer.save_pretrained("second_save")

tokenizer = AutoTokenizer.from_pretrained("./second_save")

check_tokenizer()
```

The above code outputs:

```
['some_words']
['some_words']
['some_word', 's']
```

`first_save/tokenizer.json` includes the following entry:

```
{"id":28996,"special":false,"content":"some_word","single_word":true,"lstrip":false,"rstrip":false,"normalized":true},{"id":28997,"special":false,"content":"some_words","single_word":true,"lstrip":false,"rstrip":false,"normalized":true}
```

However, in `second_save/tokenizer.json`, the above entry is changed. Note that a value for "single_word" is changed from true to false.

```
{"id":28996,"special":false,"content":"some_word","single_word":false,"lstrip":false,"rstrip":false,"normalized":true},{"id":28997,"special":false,"content":"some_words","single_word":false,"lstrip":false,"rstrip":false,"normalized":true}
```

## Expected behavior

```
['some_words']
['some_words']
['some_words']
```

Ah that's interesting! Might be of interest to @SaulLu <|||||>Edited:
The problem seems to only occur in `FastTokenizer`, the slow version works fine, 
So the problem I mentioned below has no direct relation to this issue.

@kosuke1701 You can try using your original method but add `use_fast=False` argument to `from_pretrained`, like so:
```
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased", use_fast=False)
```
The output is also correct:
```
['some_words']
['some_words']
['some_words']
```

---
This is an issue related to #13483 and #13489. @LysandreJik @SaulLu

The `add_tokens` method just simply converts `AddedToken` object into `str` and then stores it in `tokenizer.unique_no_split_tokens`, while the `save_pretrained` method only retrieves added tokens in `tokenizer.special_tokens_map_extended` so these added tokens will never be saved.  

Besides, even if `save_pretrained` method also retrieves tokens from `tokenizer.unique_no_split_tokens`, attributes that `AddedToken` provides such as `single_word` still lose because tokens have been already converted to `str` at the first step of `add_tokens`.

To OP, the following code would work:
```
import torch
import transformers
from transformers import AutoTokenizer, BertTokenizer, BertModel

from tokenizers import AddedToken

print('PyTorch version:', torch.__version__, ' transformers version:', transformers.__version__)

# Load tokenizer and add tokens.
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
new_vocab = [AddedToken("some_word", single_word=True), AddedToken("some_words", single_word=True)]
tokenizer.add_special_tokens({"additional_special_tokens":new_vocab})

def check_tokenizer():
    print(tokenizer.convert_ids_to_tokens(tokenizer.encode("some_words", add_special_tokens=False)))

check_tokenizer()

# Save and reload tokenizer
tokenizer.save_pretrained("first_save")

tokenizer = AutoTokenizer.from_pretrained("./first_save")

check_tokenizer()

# Save and reload tokenizer again
tokenizer.save_pretrained("second_save")

tokenizer = AutoTokenizer.from_pretrained("./second_save")

check_tokenizer()
```

Output:
```
['some_words']
['some_words']
['some_words']
```<|||||>@qqaatw Your code works for me! Thanks a lot :)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>It is indeed interesting! 

And thank you very much for all the analysis and the fix @qqaatw! I'm putting it on my todo list to try to find something that would avoid this undesirable behavior.