Test with Pydantic V2
### Feature request

Pydantic V2 is about to be release. There is Pydantic 2.0b3 pre-release version available already https://pypi.org/project/pydantic/2.0b3/

Please, test transformers with Pydantic V2.

There is a special tool that could help with migrating the code base to Pydantic V2 https://github.com/pydantic/bump-pydantic/

### Motivation

Pydantic V2 is known to break things and it deprecates a lot of things, see https://errors.pydantic.dev/v2.0/migration.

Why upgrading? Pydantic is known to be 5-50x faster than Pydantic V1 according to https://docs.pydantic.dev/latest/blog/pydantic-v2-alpha/. This alone looks really beneficial for `transformers`. Apart from that Pydantic V2 brings a lot of new features, see the link above.

### Your contribution

Please, don't hesitate asking for help in [Pydantic Discussions](https://github.com/pydantic/pydantic/discussions) section and/or [report any issues](https://github.com/pydantic/pydantic/issues) encountered in the process.
Hi @lig 

Thanks again for the PR #24596. Really appreciated.

Regarding using Pydantic V2, I am afraid that the involved places are not directly in `transformers` codebase.

For example, in 

https://github.com/huggingface/transformers/pull/24596#issuecomment-1615176591

it shows 

```bash
2023-06-30T20:07:31.9883431Z  > [19/19] RUN python3 -c "from deepspeed.launcher.runner import main":
2023-06-30T20:07:31.9883916Z 1.621     from deepspeed.runtime.zero.config import DeepSpeedZeroConfig
2023-06-30T20:07:31.9884613Z 1.621   File "/usr/local/lib/python3.8/dist-packages/deepspeed/runtime/zero/config.py", line 76, in <module>
2023-06-30T20:07:31.9885116Z 1.621     class DeepSpeedZeroConfig(DeepSpeedConfigModel):
2023-06-30T20:07:31.9885814Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_model_construction.py", line 171, in __new__
2023-06-30T20:07:31.9886256Z 1.621     set_model_fields(cls, bases, config_wrapper, types_namespace)
2023-06-30T20:07:31.9886812Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_model_construction.py", line 361, in set_model_fields
2023-06-30T20:07:31.9887329Z 1.621     fields, class_vars = collect_model_fields(cls, bases, config_wrapper, types_namespace, typevars_map=typevars_map)
2023-06-30T20:07:31.9888039Z 1.621   File "/usr/local/lib/python3.8/dist-packages/pydantic/_internal/_fields.py", line 112, in collect_model_fields
2023-06-30T20:07:31.9888950Z 1.621     raise NameError(f'Field "{ann_name}" has conflict with protected namespace "{protected_namespace}"')
2023-06-30T20:07:31.9889546Z 1.621 NameError: Field "model_persistence_threshold" has conflict with protected namespace "
```
which indicates `/usr/local/lib/python3.8/dist-packages/deepspeed/runtime/zero/config.py` using `pydantic`.

It's the 3rd party libraries using pydantic have to do something in order to be run with pydantic V2. Right now, `transformers` can only pin v1 and wait.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.