Problems with running 'run_lm_finetuning.py' with bert
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
My rough modification configuration (local)
    root_path = "F://IdeaProjects/transformers"
    bert_path = "F://BERT/chinese_L-12_H-768_A-12" （Downloaded from the 'https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip'）
    ## Required parameters
    parser.add_argument("--train_data_file", default=os.path.join(root_path, "data/train.txt"),...)
    parser.add_argument("--output_dir", default=os.path.join(root_path, "output")...)
    ## Other parameters
    parser.add_argument("--eval_data_file", default=os.path.join(root_path, "data/dev.txt"),...)
    parser.add_argument("--model_type", default="bert"...)
    parser.add_argument("--model_name_or_path", default=bert_path, ...)
    parser.add_argument("--mlm", action='store_true', default=True,...)
    parser.add_argument("--mlm_probability", type=float, default=0.15,...)
    parser.add_argument("--config_name", default=os.path.join(bert_path, "bert_config.json"),..)
    parser.add_argument("--tokenizer_name", default=bert_path, ...)

  
- [ Is my configuration correct?] 
 I changed this line, 'model = model_class.from_pretrained(args.model_name_or_path,
from_tf=True,#bool('.ckpt' in args.model_name_or_path),' Because my filename does not have this character（'.ckpt'）.

Then running 'run_lm_finetuning.py' causes the problem in the 'modeling_bert.py' file of the 'load_tf_weights_in_bert' method：
     'BertOnlyMLMHead' object has no attribute 'bias'

  
    
@LysandreJik do we support this one?<|||||>We do support BERT in `run_lm_finetuning`, however, we do not support loading BERT checkpoints from the original BERT implementation.

If you wish to load a checkpoint that was pre-trained/fine-tuned using the original implementation (which seems to be what you're doing here), you can first convert to our implementation using [convert_bert_original_tf_checkpoint_to_pytorch](https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py), it will then be usable by `run_lm_finetuning`.

If you wish to use TensorFlow with the outputted model, you can use the script [convert_pytorch_checkpoint_to_tf2](https://github.com/huggingface/transformers/blob/master/transformers/convert_pytorch_checkpoint_to_tf2.py) which will convert the pytorch model back to tensorflow 2.<|||||>@LysandreJik - not related to fine-tuning but converting one of the fine-tuned (rum_lm_finetuning.py) model to tensorflow checkpoint. 

Here is the command I used:
bin/python3.6 convert_pytorch_checkpoint_to_tf2.py --tf_dump_path="../tf_test/" --model_type="bert" --pytorch_checkpoint_path="../pytorch_model.bin" --config_file='../config.json'

However, it was throwing the below error (log and stack trace)

 Converting model type 1/1 bert
 Converting checkpoint 1/15: ../pytorch_model.bin - model_type bert

Building TensorFlow model from configuration: {
  "attention_probs_dropout_prob": 0.1,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "is_decoder": false,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "num_labels": 2,
  "output_attentions": true,
  "output_hidden_states": true,
  "output_past": true,
  "pruned_heads": {},
  "torchscript": false,
  "type_vocab_size": 2,
  "use_bfloat16": false,
  "vocab_size": 28996
}

Traceback (most recent call last):
  File "/home/imagen/skc/bert/transformers-2.2.1/transformers/convert_pytorch_checkpoint_to_tf2.py", line 248, in <module>
    only_convert_finetuned_models=args.only_convert_finetuned_models)
  File "/home/imagen/skc/bert/transformers-2.2.1/transformers/convert_pytorch_checkpoint_to_tf2.py", line 194, in convert_all_pt_checkpoints_to_tf
    compare_with_pt_model=compare_with_pt_model)
  File "/home/imagen/skc/bert/transformers-2.2.1/transformers/convert_pytorch_checkpoint_to_tf2.py", line 115, in convert_pt_checkpoint_to_tf
    tf_model = load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path)
  File "/home/imagen/skc/environments/.virtualenvs/lstm_dev_tf2x/lib/python3.6/site-packages/transformers/modeling_tf_pytorch_utils.py", line 82, in load_pytorch_checkpoint_in_tf2_model
    return load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys)
  File "/home/imagen/skc/environments/.virtualenvs/lstm_dev_tf2x/lib/python3.6/site-packages/transformers/modeling_tf_pytorch_utils.py", line 145, in load_pytorch_weights_in_tf2_model
    assert name in pt_state_dict, "{} not found in PyTorch model".format(name)
**AssertionError: cls.seq_relationship.weight not found in PyTorch model**

Can you please explain what did go wrong with the conversion? This is one of the BERT-base fine-tuned model. <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
