New LlamaTokenizer "fast" version takes 90s to load on 5900x with nvme
### System Info

Transformer [head]
Cuda 11.8
Pytorch nightly 2.1
Ubuntu 22.04

### Reproduction

Is it normal for the new default "faster" LlamaTokenizer to load so slowly on a fairly new cpu? Imagine the load time on a 2018 Intel xeon. 

Model is a llama 7b converted to HF using latest script within transformer/model/llama head. 

Each cold load takes ~90s. 

```

# this will take 90s to load on 5900x + nvme
# load llama 7b model converted from facebook to hf
tokenizer = AutoTokenizer.from_pretrained(base_model, use_fast=True)
```


### Expected behavior

Load in seconds. 
This is to convert the slow tokenizer to the fast format (which the conversion script should do once and for all @ArthurZucker ). You should do `tokenizer.save_pretrained(some_path)` and then copy the fast tokenizer file in the folder where you have your converted LLaMA model to avoid having the slowdown more than once as a workaround @diegomontoya until we fix the conversion script.<|||||>Yep the llama conversion script reactor happened before the llama fast tokenizer was around. Will open a PR to save the fast <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.