Slow Tokenizer adds whitespace after special token
### System Info

Python 3.10.6
Transformers 4.31.0
<class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>


### Who can help?

@ArthurZucker @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer
import transformers

tokenizer = AutoTokenizer.from_pretrained(
	"../models/llama-2-7b",
	use_fast=False,
)

txt="this is one sentence." + tokenizer.eos_token + "this is another sentence." + tokenizer.eos_token + "this is the third sentence." + tokenizer.eos_token

txt_encoded = tokenizer.encode(txt, add_special_tokens=False)
txt_encoded_decoded = tokenizer.decode(txt_encoded)
txt_encoded_decoded_spaces_false = tokenizer.decode(txt_encoded, spaces_between_special_tokens=False)

print(transformers.__version__)
print(tokenizer.__class__)

print(f"INPUT:\n{txt}\n")
print(f"ROUNDTRIP:\n{txt_encoded_decoded}\n")
print(f"ROUNDTRIP w/ spaces_between_special_tokens=F:\n{txt_encoded_decoded}\n")
```

**Output**:
```
You are using the legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at https://github.com/huggingface/transformers/pull/24565
4.31.0
<class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>
INPUT:
this is one sentence.</s>this is another sentence.</s>this is the third sentence.</s>

ROUNDTRIP:
 this is one sentence.</s> this is another sentence.</s> this is the third sentence.</s>

ROUNDTRIP w/ spaces_between_special_tokens=F:
 this is one sentence.</s> this is another sentence.</s> this is the third sentence.</s>
```


### Expected behavior

`txt == txt_encoded_decoded`

I expect `text` to be the same as `decode(encode(text))`, however a whitespace is added after each special token (`</s>`). From what I saw in previous issues, `spaces_between_special_tokens=F` should change that but it does not, whitespaces are still there.

What am I missing?

Thank you for your help and apologies in advance, this issue seems to come up quite often and I spent quite some time going through issues in this repo but nothing solved it for me.

Hey! My first suggestion would be to not use the legacy behaviour by setting `legacy = False` when you initialize the tokenizer. 
Second, the `txt_encoded == txt_encoded_decoded` assumption is not always true for all tokenizers. In this case, the decoding adds an extra space, maybe because it is based on the previous legacy behaviour. Will investigate<|||||>> My first suggestion would be to not use the legacy behaviour by setting `legacy = False` when you initialize the tokenizer.

thanks! I tried that though and it did not change the output<|||||>Ok, the same issue exists with the fast version, but the problem is with the encoding that adds extra spaces between the special tokens.... It's a mess haha<|||||>@ArthurZucker
Sorry I can't understand when and why we need to set `legacy=False` , Could you exlpain？
I run the code as follows:
```python
    txt = "one more thing" + "<s>" + "traditionally" + "<s>"
    tokenizer1 = LlamaTokenizer.from_pretrained(
        "./resources/models/llama-2-7b-hf", legacy=True, use_fast=False
    )
    tokenizer2 = LlamaTokenizer.from_pretrained(
        "./resources/models/llama-2-7b-hf", legacy=False, use_fast=False
    )

    t1 = tokenizer1.tokenize(txt)
    t2 = tokenizer2.tokenize(txt)

```
Then I got:
```
t1:['▁one', '▁more', '▁thing', '<s>', '▁tradition', 'ally', '<s>']
t2:['▁one', '▁more', '▁thing', '<s>', 'tradition', 'ally', '<s>']
```
The word starting with a `▁` usually means the start of a new word (as when comparing `▁`more and `ally`).
Even though we don't add a space before "traditionally", it is still considered a new word.
So, seems `tokenizer2` is meaningful?
 <|||||>No, words starting with `_` means that these word have a space before them, and thus the token is `_tradition`. While `tradition` is a different token. If you read the documentation that points to the PR  #24565, there is a similar example. 
What's important to understand is the concept of `added tokens`. 

Most often, sentencepiece tokenizers have a vocabulary, but some tokens are added afterwards. This happens with t5 for example. In `transformers`, we do not modify the underlying sentencepiece object. But we still support adding tokens. 

Now imagine if `thin` is part of the sentencpiece vocab, but not `_thin`. If `thin` appears next to a work like `thinking`, is will be tokenized as [`_`, `thin`, `king`], not  [`_`, `thin`, `_king`]. The same applies for any tokens that are originally part of the sentencepiece model. 

In `transformers` all `special tokens` are kind of added to the vocabulary, so we want to reproduce the behaviour and not add extra space. 

PS: please refrain from asking something pretty much unrelated. If you have a question (not a bug) feel free to post it on [the discussion forum](https://discuss.huggingface.co/)
