transformers python module “tokenizers” version is not matching with FastChat project “tokenizers” 
### System Info


Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.28.0.dev0
- Platform: Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.16
- Huggingface_hub version: 0.13.3
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1、When I installed FastChat which need to install the lastest main brach of huggingface/transformers , I found that
https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/tokenization_llama_fast.py  which tokenizers version ( require_version("tokenizers>=0.13.3") ) is not matching the latest main branch of FastChat tokenizers version:
```
https://github.com/lm-sys/FastChat/blob/main/pyproject.toml
dependencies = [
    "accelerate", "fastapi", "gradio==3.23", "markdown2[all]", "numpy",
    "requests", "sentencepiece", **"tokenizers==0.12.1",**
    "torch", "uvicorn", "wandb",
    "transformers @ git+https://github.com/huggingface/transformers.git"
]
```
2、So ，FastChat current version（0.1.4） needs to match which transformers version and tokenizers version?

Errors:
```
Loading base model
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 41/41 [00:26<00:00,  1.54it/s]
Loading delta
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:19<00:00,  6.47s/it]
Traceback (most recent call last):
  File "/home/jiagy/transformers/src/transformers/utils/import_utils.py", line 1125, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/root/miniconda3/envs/fast-chat/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/jiagy/transformers/src/transformers/models/llama/tokenization_llama_fast.py", line 19, in <module>
    require_version("tokenizers>=0.13.3")
  File "/home/jiagy/transformers/src/transformers/utils/versions.py", line 117, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/home/jiagy/transformers/src/transformers/utils/versions.py", line 50, in _compare_versions
    raise ImportError(
ImportError: tokenizers>=0.13.3 is required for a normal functioning of this module, but found tokenizers==0.12.1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/miniconda3/envs/fast-chat/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/miniconda3/envs/fast-chat/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jiagy/FastChat/fastchat/model/apply_delta.py", line 49, in <module>
    apply_delta(args.base_model_path, args.target_model_path, args.delta_path)
  File "/home/jiagy/FastChat/fastchat/model/apply_delta.py", line 19, in apply_delta
    delta_tokenizer = AutoTokenizer.from_pretrained(delta_path)
  File "/home/jiagy/transformers/src/transformers/models/auto/tokenization_auto.py", line 691, in from_pretrained
    tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate)
  File "/home/jiagy/transformers/src/transformers/models/auto/tokenization_auto.py", line 392, in tokenizer_class_from_name
    return getattr(module, class_name)
  File "/home/jiagy/transformers/src/transformers/utils/import_utils.py", line 1115, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/jiagy/transformers/src/transformers/utils/import_utils.py", line 1127, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.tokenization_llama_fast because of the following error (look up to see its traceback):
tokenizers>=0.13.3 is required for a normal functioning of this module, but found tokenizers==0.12.1.
```

### Expected behavior

transformer tokenizers version is matching with FastChat  tokenizers version.
I'm not sure why you are opening the issue here. It's a problem in the dependencies of FastChat.<|||||>Because this makes me very confused about which dependency configuration should refer to when project updating so quickly<|||||>This issue can resolve by installing a specific version's transformer (with source),：
```
pip uninstall transformers
pip install git+https://github.com/huggingface/transformers@cae78c46d
```<|||||>After implementing the soln, this issue pops up
Failed to import transformers.models.llama.tokenization_llama_fast because of the following error (look up to see its traceback):
No module named 'transformers.models.llama.tokenization_llama_fast'
<|||||>pip install tokenizers==0.13.3 -- this should solve your problem