LLAMA 2 HF tokenizer len is 32001
### System Info

Installed from source (4.32.0.dev0)


```
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-13b-hf")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-13b-hf")
print(len(tokenizer)) #32001
print(model.config.vocab_size) #32000
```



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-13b-hf")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-13b-hf")
print(len(tokenizer)) #32001
print(model.config.vocab_size) #32000
```

### Expected behavior

model vocab size and tokenizer len should be 32000. It seems the padding token of the tokenizer is set to '\<unk\>'. Which is not normally the case. It's normally not set.
The actual config says that `pad_token_id=0` - so I assume this is correct?

What is interesting is that id `32000` maps to a token `'<pad>'` while the original vocab does not contain this token:
https://huggingface.co/meta-llama/Llama-2-7b-hf/raw/main/tokenizer.json

It seems this is being added somewhere in HF code?<|||||>cc @ArthurZucker <|||||>Hey! Yes this is not entirely expected, we update the slow tokenizer, but the fast version did not get the update. I'll open PRs to fix this! <|||||>@ArthurZucker jfyi I see the same behavior for slow and fast on 4.31.0

edit: correction, indeed it is not set in the tokenizer<|||||>Is there any plan to fix it?<|||||>It is fixed on all model! <|||||>@ArthurZucker  Does the fix really resolve this? I pip installed `transformers` from Github, and there is still a mismatch.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# Using fast tokenizer by default
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
print(len(tokenizer)) # 32001
print(model.config.vocab_size) # 32000
print(tokenizer.get_added_vocab()) # {'<pad>': 32000}
```

Transformers version: `4.32.0.dev0`<|||||>Will check asap! Might be the fast tokenizer that did not get updated.<|||||>Sorry but no, I cannot reproduce your issue. I do not know if you maybe did not update cached files but I have this: 
```python 
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
>>> len(tokenizer)
32000
>>> print(tokenizer.get_added_vocab())
{}
```<|||||>Thanks for confirming. You're right, it's because of the locally cached files. Downloading them from huggingface and re-running the code now works fine.