Loading from ckpt is not possible for bert, neither tf to pytorch conversion works in 2.1.1
## 🐛 Bug

<!-- Important information -->
- I am trying to load a BERT model (for simplicity assume the original uncased-base from google's repo) using instructions in: https://github.com/huggingface/transformers/blob/ae1d03fc51bb22ed59517ee6f92c560417fdb049/transformers/modeling_tf_utils.py#L195
and more specifically:
`self.tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')`
`config = BertConfig.from_json_file('data/uncased_L-12_H-768_A-12/config.json')`
`self.model = TFBertForSequenceClassification.from_pretrained(                                                                   pretrained_model_name_or_path='data/uncased_L-12_H-768_A-12/model.ckpt.index', config=config, from_pt=True)`

- This fails as expected since you need to change line https://github.com/huggingface/transformers/blob/ae1d03fc51bb22ed59517ee6f92c560417fdb049/transformers/modeling_tf_utils.py#L225
with this `elif os.path.isfile(os.path.join(pretrained_model_name_or_path, TF_WEIGHTS_NAME)):`
and set from_pt=False.
- Even then it fails in https://github.com/huggingface/transformers/blob/ae1d03fc51bb22ed59517ee6f92c560417fdb049/transformers/modeling_tf_utils.py#L274 with some tf notImplementedError.

Then I decided to use the converter and turn the tf ckpt into pytorch:
https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py
This seems to do the conversion correctly but then fails when loading it (I follow precisely the same steps from https://github.com/huggingface/transformers/issues/457#issuecomment-518403170 )
it fails with `AssertionError: classifier.weight not found in PyTorch model`.

So if I am not missing sth, at this point it does not seem possible to load somehow a tf ckpt?
Would it make sense to convert ckpt to h5 and use that?
Thanks!


Model I am using (Bert, XLNet....):
Bert
Language I am using the model on (English, Chinese....):
English
The problem arise when using:
* [ x] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1. convert_tf_checkpoint_to_pytorch('data/uncased_L-12_H-768_A-12/model.ckpt',
                                     'data/uncased_L-12_H-768_A-12/config.json',
                                     'data/uncased_L-12_H-768_A-12/pytorch_model.bin')
2. model = TFBertForSequenceClassification.from_pretrained('data/uncased_L-12_H-768_A-12/', from_pt=True)

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Debian Linux
* Python version: 3.6.8
* PyTorch version: 1.3.0
* PyTorch Transformers version (or branch): 2.1.1
* Using GPU ? yes
* Distributed of parallel setup ?
* Any other relevant information:
tf 2.0
## Additional context

<!-- Add any other context about the problem here. -->

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
