🚨🚨 🚨🚨 [WIP][Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 
# What does this PR do?
Adresses a lot of issues related to `add_tokens`, also adds more refine testing to make sure this does not happen again. 

- Adding a token with `add_tokens` ignores the arguments if the token is an `AddedToken`. reported in #20734 and #14770, #21120, #16334 
- Adding a token does not automatically adds it to the `unique_no_split_token`. Reported in #23818 , #23851, #11531  but also #23250. Also linked to #22940 , should allow us to re-factor the way T5 tokenizes the inputs (convert_token_to_ids should not have a bunch of regex for special tokens) (also #9747)
- Adding a token that is already in the vocabulary does not add it. This is debatable, but if someone explicitly adds it, it means he does not want to split it. Reported in #23459 
- There is no support for `single_word` in `slow`. Reported in #14770 
-  Initialising a model from a vocab file does not initialize the Trie. `from_pretrained` calls `added_tokens = tokenizer.sanitize_special_tokens()` which is when the tokens are added to no_unique_split. reported in #23930

One more: #22935 
cc @Narsil for visibility


- [ ] Default `AddedTokens` class is the same for `tokenizers` and `transformers`, but in transformers since by default we use to always strip left and right....... it's a mess
- [ ] TODO Tokens that are normalized should have two versions added to the `trie()`: for spm, both `<s>` and `▁<s>` should point to the special tokens, as the prefix `▁` has always been removed for special tokens.


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23909). All of your documentation changes will be reflected on that endpoint.<|||||>TODO: the test should be for all the `TokenMixin` rather than comparing slow and fast as the behaviour should work in any case.<|||||>Is work on this still going on? I'm interested in working on tokenization.<|||||>It is! Had to focus on a new model for a while but this is close to being over <|||||>Cool! So, I think the best way to do this is that I fork your branch, and then do PRs there? Any changes you like would then propagate to this PR? Is that the preferred way of collaborating?<|||||>Haha sorry what I meant is that I’ll take care of this one probably today, tomorrow so no need to dive ! 
Otherwise forking transformers is always the best solution IMO, then add others as remotes.<|||||>Ah ok, my bad! I totally misunderstood. Good luck with the PR 😸 <|||||>Will come back to this starting next week 🔥 <|||||>Another TODO: if someone wants to `unset` one of the special token, it should be removed from the added_tokens_encoder and added_tokens_decoder. Some things to think about here.
Also `added_token_encoder` should just always be linked to the decoder, which will be the only one that exists