GPT-NeoX missing Tokenizer
### System Info

```shell
- `transformers` version: 4.21.0.dev0
- Platform: Linux-5.13.0-44-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes, deepspeed
```


### Who can help?

@patil-suraj @SaulLu 

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When I try to load the model using the following script, it hangs and tells you that the tokenizer GPTNeoXTokenizer does not exist.

```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
```
This does not happen when using the "Fast" version.

### Expected behavior

```shell
Model should work, tokenizer should load.
```

Error I am receiving:
```
ImportError: cannot import name 'GPTNeoXTokenizer' from 'transformers' (/opt/conda/lib/python3.8/site-packages/transformers/init.py)
```<|||||>Just tried your code sample and it works fine on my side. Are you sure you have `tokenizers` installed in your env? GPT-Neo-X does not have a slow tokenizer, so it requires this library.<|||||>> Just tried your code sample and it works fine on my side. Are you sure you have `tokenizers` installed in your env? GPT-Neo-X does not have a slow tokenizer, so it requires this library.

Just checked:
tokenizers             0.12.1<|||||>Only the fast tokenizer is available for GPT-NeoX-20B.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Hello, @mrseeker did you resolve your issue? I have same problem, trying to get working version of transformers...<|||||>I have the exact same issue.

```
│ /usr/local/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:699 in     │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   696 │   │   │   │   tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate)     │
│   697 │   │   │                                                                                  │
│   698 │   │   │   if tokenizer_class is None:                                                    │
│ ❱ 699 │   │   │   │   raise ValueError(                                                          │
│   700 │   │   │   │   │   f"Tokenizer class {tokenizer_class_candidate} does not exist or is n   │
│   701 │   │   │   │   )                                                                          │
│   702 │   │   │   return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *input 
```<|||||>For those having an issue with this, the resolution is as follows:

Use AutoTokenizer**Fast**, AutoTokenizer is not supported by NeoX.<|||||>if you use fastchat,modify fastchat/model/mode_adapter.py like this


def load_model(self, model_path: str, from_pretrained_kwargs: dict):
        tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=True)
        model = AutoModelForCausalLM.from_pretrained(
            model_path, low_cpu_mem_usage=True,trust_remote_code=True, **from_pretrained_kwargs
        )
        return model, tokenizer


will fix this issue