Issue when load pretrained weights
I got the following error when running 

AutoModelWithLMHead.from_pretrained("bert-base-chinese")


OSError: Can't load weights for 'bert-base-chinese'. Make sure that:

- 'bert-base-chinese' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'bert-base-chinese' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.
Cannot reproduce. The command:

```python 
from transformers import AutoModelWithLMHead
model = AutoModelWithLMHead.from_pretrained("bert-base-chinese")
```

works fine on master. Can you update to v3.0.2 `pip install --upgrade transformers` and check again? :-) <|||||>> Cannot reproduce. The command:
> 
> ```python
> from transformers import AutoModelWithLMHead
> model = AutoModelWithLMHead.from_pretrained("bert-base-chinese")
> ```
> 
> works fine on master. Can you update to v3.0.2 `pip install --upgrade transformers` and check again? :-)

Still not work for me. I tried to download the weights directly, but another error occurred...
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte<|||||>Can you please post your environment info here `python src/transformers/commands/env.py`?<|||||>I'm seeing the same error when trying to load a GTP2 checkpoint model (using both `GPT2Model` and `AutoModel`):

```
model = GPT2Model.from_pretrained('./test_01/test_01.index', from_tf=True) # throws UnicodeDecodeError
model = GPT2Model.from_pretrained('./test_01/test_01.index') # throws UnicodeDecodeError
model = AutoModel.from_pretrained('./test_01/test_01.index', from_tf=True) # throws UnicodeDecodeError
model = AutoModel.from_pretrained('./test_01/test_01.index') # throws UnicodeDecodeError
```

I could probably try every possible variation of loading that model and hit the same error.

I've also used checkpoint models that, in theory, should work.

If I use `GPT2LMHeadModel.from_pretrained('gpt2-medium')` (or any thing that allows me to load a model by name) it works fine.

My env:

```
- `transformers` version: 3.0.2
- Platform: macOS-10.15.5-x86_64-i386-64bit
- Python version: 3.8.3
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Not sure
```
<|||||>@elbowdonkey, 

can you try just running:
```python 
model = GPT2Model.from_pretrained('./test_01/", from_tf=True)
```

where the relevant files can be found in `test_01`?<|||||>I get a different error:

```python
model = GPT2Model.from_pretrained("./test_02/", from_tf=True)
2020-08-10 12:32:10.047629: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-08-10 12:32:10.064526: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8eb76b8f80 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-10 12:32:10.064543: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-08-10 12:32:10.071086: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2020-08-10 12:32:27.584426: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open ./test_02/pytorch_model.bin: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/--/.pyenv/versions/3.8.3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 808, in from_pretrained
    model = load_tf2_checkpoint_in_pytorch_model(model, resolved_archive_file, allow_missing_keys=True)
  File "/Users/--/.pyenv/versions/3.8.3/lib/python3.8/site-packages/transformers/modeling_tf_pytorch_utils.py", line 261, in load_tf2_checkpoint_in_pytorch_model
    tf_model.load_weights(tf_checkpoint_path, by_name=True)
  File "/Users/--/.pyenv/versions/3.8.3/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2204, in load_weights
    with h5py.File(filepath, 'r') as f:
  File "/Users/--/.pyenv/versions/3.8.3/lib/python3.8/site-packages/h5py/_hl/files.py", line 406, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/Users/--/.pyenv/versions/3.8.3/lib/python3.8/site-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
```

The model I'm trying to use is a model that was converted from a checkpoint to a pytorch model. I have no idea what kind of checkpoint model it was (it has several files: `checkpoint` and `vocab.bpe`, `hparams.json`, and `test_02.data-00000-of-00001`, among others.)<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hi,

I would like to request that this ticket be opened back up. I'm having the same issue but with the default pretrained model:

```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~\Anaconda3\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    654                 if resolved_archive_file is None:
--> 655                     raise EnvironmentError
    656             except EnvironmentError:

OSError: 

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-81-7e9fe0224671> in <module>
      3 #Load AutoModel from huggingface model repository
      4 tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
----> 5 model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens", from_tf=True)

~\Anaconda3\lib\site-packages\transformers\modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    500         for config_class, model_class in MODEL_MAPPING.items():
    501             if isinstance(config, config_class):
--> 502                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    503         raise ValueError(
    504             "Unrecognized configuration class {} for this kind of AutoModel: {}.\n"

~\Anaconda3\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    660                     f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a file named one of {WEIGHTS_NAME}, {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME}.\n\n"
    661                 )
--> 662                 raise EnvironmentError(msg)
    663 
    664             if resolved_archive_file == archive_file:

OSError: Can't load weights for 'sentence-transformers/bert-base-nli-mean-tokens'. Make sure that:

- 'sentence-transformers/bert-base-nli-mean-tokens' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sentence-transformers/bert-base-nli-mean-tokens' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.
```

**To reproduce:**
```
from transformers import AutoTokenizer, AutoModel

#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens", from_tf=True)
```

My env:

- `transformers` version: 3.0.2
- Platform: Window 10 Enterprise, version 1909, 16GB RAM, 64 Bit OS, x64-based processor
- Python version: 3.8.3
- Torch version: 1.6.0+cpu<|||||>Hey @Ecanlilar,

This model exists only in PT so either you do:

```python
from transformers import AutoTokenizer, AutoModel

#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
```

or 

```python
from transformers import AutoTokenizer, TFAutoModel

#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
model = TFAutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens", from_pt=True)
```<|||||>> Hey @Ecanlilar,
> 
> This model exists only in PT so either you do:
> 
> ```python
> from transformers import AutoTokenizer, AutoModel
> 
> #Load AutoModel from huggingface model repository
> tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
> model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
> ```
> 
> or
> 
> ```python
> from transformers import AutoTokenizer, TFAutoModel
> 
> #Load AutoModel from huggingface model repository
> tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens")
> model = TFAutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens", from_pt=True)
> ```

This isn't working for me. I'm using the latest version of the transformers library (4.10.2). I'm getting the same error as Ecanlilar.