Llama Tokenizer Unexpectedly Producing Unknown Token
### System Info

- `transformers` version: 4.29.2
- Platform: Linux-5.19.0-1027-aws-x86_64-with-glibc2.31
- Python version: 3.10.8
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)

### Who can help?

@ArthurZucker @younesbelkada I am trying to use special tokens with the LlamaTokenizer in Transformers 4.31.0 and with certain configurations of input, the tokenizer is returning a token id of 0 corresponding to the unknown token. For example, I have added the special token "<REPR_END>", and if I pass that through the tokenizer to get [1, 32003] which is good. Additionally if I pass the word "inform" through the tokenizer, I get [1, 1871], which is also good.

However, if I pass "<REPR_END>inform" through the tokenizer, I get [1, 32003, 0] which does not make sense. If I try this exact same input in Transformers 4.29.2, I get [1, 32003, 1871] which is correct.

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction
```python 
from transformers.models.llama.tokenization_llama import LlamaTokenizer

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-13b-hf", use_auth_token=...)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.add_tokens(['<TARGET_BEGIN>', '<TARGET_END>', '<REPR_BEGIN>', '<REPR_END>'], special_tokens=True)

print(tokenizer("<REPR_END>inform")
```

### Expected behavior

I should expect to get the output [1, 32003, 1871] but I do not. I instead get [1, 32003, 0]
Hey! 👋🏻  Thanks for providing a reproduction script. I suspect that you do not have `tokenizers` installed, since when I use `use_fast = True` (which is the default if you have tokenizers) the issue is not present. 
Now, this behaviour is expected: 
- Quick fix, use `legacy=True` when initialising the tokenizer: `tokenizer = LlamaTokenizer.from_pretrained("meta-llama/Llama-2-13b-hf", legacy = True)`
- Other quick fix, use the fast tokenizer ( `pip install tokenizers`)

This is a very nice catch otherwise! The issue is that [`in`, `form`]  should be the tokenization of `inform`, but when we use the hack around sentencepiece, we actually just output [`inform`] which is not recognised as a token. Also if you pass `"<REPR_END> inform"` the extra space is automatically strip by default. This is also gonna be fixed. 

cc @Narsil I think I'll implement handling the `add_dummy_prefix = False` parameter. As pointed out somewhere else, our decoding function is also broken for Llama (it add extra spaces). 