AttributeError: 'BertOnlyMLMHead' object has no attribute 'bias'
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

I was trying to load a RuBERT model from [DeepPavlov](http://docs.deeppavlov.ai/en/master/features/models/bert.html) but ran into this error. The model is in TensorFlow and the code I used to load it is:
```
config = BertConfig.from_json_file('rubert_cased_L-12_H-768_A-12_v2/bert_config.json')
model = BertForMaskedLM.from_pretrained('rubert_cased_L-12_H-768_A-12_v2/bert_model.ckpt.index', from_tf=True, config=config)
model.eval()
```
The error message is the following:
```
AttributeError                            Traceback (most recent call last)
<ipython-input-150-74d68b4b5d71> in <module>
      1 config = BertConfig.from_json_file('rubert_cased_L-12_H-768_A-12_v2/bert_config.json')
----> 2 model = BertForMaskedLM.from_pretrained('rubert_cased_L-12_H-768_A-12_v2/bert_model.ckpt.index', from_tf=True, config=config)
      3 model.eval()

c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    352             if resolved_archive_file.endswith('.index'):
    353                 # Load from a TensorFlow 1.X checkpoint - provided by original authors
--> 354                 model = cls.load_tf_weights(model, config, resolved_archive_file[:-6])  # Remove the '.index'
    355             else:
    356                 # Load from our TensorFlow 2.0 checkpoints

c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\transformers\modeling_bert.py in load_tf_weights_in_bert(model, config, tf_checkpoint_path)
     90                 pointer = getattr(pointer, 'weight')
     91             elif l[0] == 'output_bias' or l[0] == 'beta':
---> 92                 pointer = getattr(pointer, 'bias')
     93             elif l[0] == 'output_weights':
     94                 pointer = getattr(pointer, 'weight')

c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\torch\nn\modules\module.py in __getattr__(self, name)
    533                 return modules[name]
    534         raise AttributeError("'{}' object has no attribute '{}'".format(
--> 535             type(self).__name__, name))
    536 
    537     def __setattr__(self, name, value):

AttributeError: 'BertOnlyMLMHead' object has no attribute 'bias'
```

I've also tried to load the [official BERT models from Google](https://github.com/google-research/bert/blob/master/multilingual.md) and got the same result.
> ## ❓ Questions & Help
> I was trying to load a RuBERT model from [DeepPavlov](http://docs.deeppavlov.ai/en/master/features/models/bert.html) but ran into this error. The model is in TensorFlow and the code I used to load it is:
> 
> ```
> config = BertConfig.from_json_file('rubert_cased_L-12_H-768_A-12_v2/bert_config.json')
> model = BertForMaskedLM.from_pretrained('rubert_cased_L-12_H-768_A-12_v2/bert_model.ckpt.index', from_tf=True, config=config)
> model.eval()
> ```
> 
> The error message is the following:
> 
> ```
> AttributeError                            Traceback (most recent call last)
> <ipython-input-150-74d68b4b5d71> in <module>
>       1 config = BertConfig.from_json_file('rubert_cased_L-12_H-768_A-12_v2/bert_config.json')
> ----> 2 model = BertForMaskedLM.from_pretrained('rubert_cased_L-12_H-768_A-12_v2/bert_model.ckpt.index', from_tf=True, config=config)
>       3 model.eval()
> 
> c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
>     352             if resolved_archive_file.endswith('.index'):
>     353                 # Load from a TensorFlow 1.X checkpoint - provided by original authors
> --> 354                 model = cls.load_tf_weights(model, config, resolved_archive_file[:-6])  # Remove the '.index'
>     355             else:
>     356                 # Load from our TensorFlow 2.0 checkpoints
> 
> c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\transformers\modeling_bert.py in load_tf_weights_in_bert(model, config, tf_checkpoint_path)
>      90                 pointer = getattr(pointer, 'weight')
>      91             elif l[0] == 'output_bias' or l[0] == 'beta':
> ---> 92                 pointer = getattr(pointer, 'bias')
>      93             elif l[0] == 'output_weights':
>      94                 pointer = getattr(pointer, 'weight')
> 
> c:\users\milin\appdata\local\programs\python\python36\lib\site-packages\torch\nn\modules\module.py in __getattr__(self, name)
>     533                 return modules[name]
>     534         raise AttributeError("'{}' object has no attribute '{}'".format(
> --> 535             type(self).__name__, name))
>     536 
>     537     def __setattr__(self, name, value):
> 
> AttributeError: 'BertOnlyMLMHead' object has no attribute 'bias'
> ```
> 
> I've also tried to load the [official BERT models from Google](https://github.com/google-research/bert/blob/master/multilingual.md) and got the same result.
hey!
Have you solved this problem? I have the same problem!ROLAND JUNO-STAGE!<|||||> @lichunnan
After studying the manual more thoroughly, I found that you should [first convert the TensorFlow models to PyTorch](https://huggingface.co/transformers/converting_tensorflow_models.html) with [this script](https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py).