Cached TF files cannot be loaded with `from_pretrained` without internet connection
# 🐛 Bug

## Information
For TF models in latest version of `transformers v2.11.0`, when the internet is turned off, `from_pretrained` throws an error even when the model has previously been downloaded to cache:

```python
# does not work (wifi/internet turned off)
model = TFBertModel.from_pretrained('bert-base-uncased', local_files_only=True)
```
Error is:
```
OSError: Can't load weights for 'bert-base-uncased'. Make sure that:

- 'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'bert-base-uncased' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

```
The above error occurs regardless of how `local_files_only` is set. This can be problematic in certain settings (e.g., air-gapped networks, firewalls).

By contrast, for PyTorch models with NO internet, `from_pretrained` works perfectly and correctly loads files from cache both when `local_files_only=False` and `local_files_only=True`, which is a  useful parameter recently added in PR #2930 by @BramVanroy and merged by @LysandreJik:

```python
# works and is fast (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=True)

# works but is slow due to outgoing connection attempts (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=False)
```

## Environment Info

- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (True)


Hi! This issue should have been solved by https://github.com/huggingface/transformers/pull/5116<|||||>Hi @LysandreJik:  I just wanted to let you know that this issue is still not resolved as of v3.02.  If you ensure the model files (and vocab, etc.) exist in the cache, and then you turn off wifi/internet, model loading does not work in TensorFlow (but does work in PyTorch):

This **does not** work:
```python
# wifi off and model files exist in cache
rom transformers import *
model = TFAutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased')
# error is: Cannot load weights for distilbert-base-uncased
```

This **does** work:
```python
# wifi off and model files exist in cache
from transformers import *
model = AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased')
```
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Thanks, @LysandreJik .  I can confirm that PR #6091 resolves this issue (tested by manually applying fix to a local `transformers==3.3.0` installation).