`"transformers_version"` is not enforced
### System Info

- `transformers` version: 4.21.1
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.9.7
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.9.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes+no
- Using distributed or parallel set-up in script?: no

### Who can help?

This is a general problem with loading pretrained models in the library, not any specific model:
@sgugger @stevhliu @patrickvonplaten


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NinedayWang/PolyCoder-2.7B")
model = AutoModelForCausalLM.from_pretrained("NinedayWang/PolyCoder-2.7B")
```

### Expected behavior

The model's `config.json` specifies that this model requires version `4.23.1` here:
https://huggingface.co/NinedayWang/PolyCoder-2.7B/blob/main/config.json#L21

but if the user is trying to load this model with an older version (`4.21.1`), it still incorrectly and silently does load.

So, I would expect an error instead of a successful loading, because with an old version of `transformers`, this model produces incorrect predictions.
I prefer not being able to load the checkpoint if I don't have the required version of the library.

In other words, why do we have the `"transformers_version"` field in the `config.json`, if it is not enforced?
Thanks!

It is just informative (and saved automatically, not selected by the user), so it is by no way a minimal version required for the model.<|||||>thanks!

Don't we want a way to enforce that a certain checkpoint will be loaded only by a specific version?
Currently users may download checkpoints, load them successfully, but just get bad predictions<|||||>DO you have an explicit example of that? We maintain very strict backward/forward compatibility in the model code.<|||||>Yes,
This model `NinedayWang/PolyCoder-2.7B` must be loaded with version `>=4.23.0`,
because it depends on this PR: https://github.com/huggingface/transformers/pull/18695

<|||||>Ah, yet another argument for not adding config arguments like those. Thanks for pointing that out.<|||||>Thanks a lot!