Weights of FlaubertForQuestionAnswering not initialized from pretrained model
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

`Flaubert`

Language I am using the model on (English, Chinese ...):

`French`

The tasks I am working on is: **Fine-tune Flaubert on French-translated SQuAD**

The problem arises when using:

```
python3 ./examples/run_squad.py \
--model_type flaubert \
--model_name_or_path flaubert-base-uncased \
--do_train \
--do_eval \
--do_lower_case \
--train_file SQuAD-v1.1-train_fr_ss999_awstart2_net.json \
--predict_file SQuAD-v1.1-dev_fr_ss999_awstart2_net.json \
--learning_rate 3e-5 \
--num_train_epochs 2 \
--max_seq_length 384 \
--doc_stride 128 \
--output_dir output \
--per_gpu_eval_batch_size=3 \
--per_gpu_train_batch_size=3
```

For some reason the downloaded weights from pre-trained model `flaubert-base-uncased` are not initialized for training:

```python-traceback
2/02/2020 15:10:53 - WARNING - __main__ -   Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False
02/02/2020 15:10:53 - INFO - transformers.configuration_utils -   loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/flaubert/flaubert_base_uncased/config.json from cache at /root/.cache/torch/transformers/d1cf66823bb82e0ef671e7bae75bf86161cbf8ca218f893bc0129599e6e40c2a.e40562626242ae71bf0ce9aa0832297b724c4859407a09771341048981bb3736
02/02/2020 15:10:53 - INFO - transformers.configuration_utils -   Model config FlaubertConfig {
  "amp": 1,
  "architectures": [
    "FlaubertWithLMHeadModel"
  ],
  "asm": false,
  "attention_dropout": 0.1,
  "bos_index": 0,
  "bos_token_id": 0,
  "bptt": 512,
  "causal": false,
  "clip_grad_norm": 5,
  "do_sample": false,
  "dropout": 0.1,
  "emb_dim": 768,
  "embed_init_std": 0.02209708691207961,
  "encoder_only": true,
  "end_n_top": 5,
  "eos_index": 1,
  "eos_token_ids": 0,
  "finetuning_task": null,
  "fp16": true,
  "gelu_activation": true,
  "group_by_size": true,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "id2lang": {
    "0": "fr"
  },
  "init_std": 0.02,
  "is_decoder": false,
  "is_encoder": true,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "lang2id": {
    "fr": 0
  },
  "lang_id": 0,
  "langs": [
    "fr"
  ],
  "layer_norm_eps": 1e-12,
  "layerdrop": 0.0,
  "length_penalty": 1.0,
  "lg_sampling_factor": -1,
  "lgs": "fr",
  "mask_index": 5,
  "mask_token_id": 0,
  "max_batch_size": 0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "max_vocab": -1,
  "mlm_steps": [
    [
      "fr",
      null
    ]
  ],
  "model_type": "flaubert",
  "n_heads": 12,
  "n_langs": 1,
  "n_layers": 12,
  "num_beams": 1,
  "num_labels": 2,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pad_index": 2,
  "pad_token_id": 0,
  "pre_norm": false,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "sample_alpha": 0,
  "share_inout_emb": true,
  "sinusoidal_embeddings": false,
  "start_n_top": 5,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "first",
  "summary_use_proj": true,
  "temperature": 1.0,
  "tokens_per_batch": -1,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "unk_index": 3,
  "use_bfloat16": false,
  "use_lang_emb": true,
  "vocab_size": 67542,
  "word_blank": 0,
  "word_dropout": 0,
  "word_keep": 0.1,
  "word_mask": 0.8,
  "word_mask_keep_rand": "0.8,0.1,0.1",
  "word_pred": 0.15,
  "word_rand": 0.1,
  "word_shuffle": 0
}

02/02/2020 15:10:53 - INFO - transformers.tokenization_utils -   loading file https://s3.amazonaws.com/models.huggingface.co/bert/flaubert/flaubert_base_uncased/vocab.json from cache at /root/.cache/torch/transformers/8f54ff51875f0422a9c265ab77515058f2655b901caa5f8ff19954c8a126a2fe.4dbbb80764d7ce5ea8639cef2ffdf2c6be3c491192c042bba9651d56b917d49c
02/02/2020 15:10:53 - INFO - transformers.tokenization_utils -   loading file https://s3.amazonaws.com/models.huggingface.co/bert/flaubert/flaubert_base_uncased/merges.txt from cache at /root/.cache/torch/transformers/42f0fe2cd5eebb0c450bd936b0104b27c21e33138b445e9c7124094e05df02f6.5e19e4f2e2e9e11ecde5cc44c2c65f0dc11671ff5dfcd0066699e64bbc7c5a8d
02/02/2020 15:10:53 - INFO - transformers.modeling_utils -   loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/flaubert/flaubert_base_uncased/pytorch_model.bin from cache at /root/.cache/torch/transformers/2931084022a5d35320c07628cb7de631bdefe38f0e87d5d48a9e04be799ce0ef.8a02ed26eb9bc391a8fd64b6acce3b2167eb7a01cd4365502dca3a5980918425
02/02/2020 15:11:00 - INFO - transformers.modeling_utils -   Weights of FlaubertForQuestionAnswering not initialized from pretrained model: ['qa_outputs.start_logits.dense.bias', 'qa_outputs.start_logits.dense.weight', 'qa_outputs.end_logits.dense_0.bias', 'qa_outputs.end_logits.dense_0.weight', 'qa_outputs.end_logits.LayerNorm.bias', 'qa_outputs.end_logits.LayerNorm.weight', 'qa_outputs.end_logits.dense_1.bias', 'qa_outputs.end_logits.dense_1.weight', 'qa_outputs.answer_class.dense_0.bias', 'qa_outputs.answer_class.dense_0.weight', 'qa_outputs.answer_class.dense_1.weight']
02/02/2020 15:11:00 - INFO - transformers.modeling_utils -   Weights from pretrained model not used in FlaubertForQuestionAnswering: ['pred_layer.proj.bias', 'pred_layer.proj.weight']
02/02/2020 15:11:06 - INFO - __main__ -   Training/evaluation parameters Namespace(adam_epsilon=1e-08, cache_dir='', config_name='', data_dir=None, device=device(type='cuda'), do_eval=True, do_lower_case=True, do_train=True, doc_stride=128, eval_all_checkpoints=False, evaluate_during_training=False, fp16=False, fp16_opt_level='O1', gradient_accumulation_steps=1, learning_rate=3e-05, local_rank=-1, logging_steps=500, max_answer_length=30, max_grad_norm=1.0, max_query_length=64, max_seq_length=384, max_steps=-1, model_name_or_path='flaubert-base-uncased', model_type='flaubert', n_best_size=20, n_gpu=1, no_cuda=False, null_score_diff_threshold=0.0, num_train_epochs=2.0, output_dir='output', overwrite_cache=False, overwrite_output_dir=False, per_gpu_eval_batch_size=3, per_gpu_train_batch_size=3, predict_file='SQuAD-v1.1-dev_fr_ss999_awstart2_net.json', save_steps=500, seed=42, server_ip='', server_port='', threads=1, tokenizer_name='', train_file='SQuAD-v1.1-train_fr_ss999_awstart2_net.json', verbose_logging=False, version_2_with_negative=False, warmup_steps=0, weight_decay=0.0)
02/02/2020 15:11:06 - INFO - __main__ -   Creating features from dataset file at .
100%|██████████████████████████████████████████████████████████████████████████████████████| 442/442 [00:42<00:00, 10.49it/s]
convert squad examples to features:   4%|█▉                                             | 3457/84943 [02:18<37:30, 36.21it/s]
```


## To reproduce

Steps to reproduce the behavior:

Edit `run_squad.py` to support `flaubert`:

```python
MODEL_CLASSES = {
    "bert": (BertConfig, BertForQuestionAnswering, BertTokenizer),
    "roberta": (RobertaConfig, RobertaForQuestionAnswering, RobertaTokenizer),
    "xlnet": (XLNetConfig, XLNetForQuestionAnswering, XLNetTokenizer),
    "xlm": (XLMConfig, XLMForQuestionAnswering, XLMTokenizer),
    "distilbert": (DistilBertConfig, DistilBertForQuestionAnswering, DistilBertTokenizer),
    "albert": (AlbertConfig, AlbertForQuestionAnswering, AlbertTokenizer),
    "flaubert": (FlaubertConfig, FlaubertForQuestionAnswering, FlaubertTokenizer),
}
```

I had to do some other little edits. Then execute script:

```bash
python3 ./examples/run_squad.py \
--model_type flaubert \
--model_name_or_path flaubert-base-uncased \
--do_train \
--do_eval \
--do_lower_case \
--train_file SQuAD-v1.1-train_fr_ss999_awstart2_net.json \
--predict_file SQuAD-v1.1-dev_fr_ss999_awstart2_net.json \
--learning_rate 3e-5 \
--num_train_epochs 2 \
--max_seq_length 384 \
--doc_stride 128 \
--output_dir output \
--per_gpu_eval_batch_size=3 \
--per_gpu_train_batch_size=3
```

Dataset available here: https://github.com/Alikabbadj/French-SQuAD

## Expected behavior

Load weights from pre-trained `flaubert-base-uncased` model to fine-tune on FR SQuAD train then use new trained weights to evaluate model on FR SQuAD dev. 

## Environment info
     
- `transformers` version: `transformers==2.4.1`
- Platform: `Deep Learning AMI (Ubuntu 16.04) Version 26.0 `

The Flaubert checkpoints contain the base transformer model, not the weights for question answering (similar to most checkpoints). The point of the `run_squad` script is to fine-tune the weights of the additional question answering head to the specific task (french squad in your case).<|||||>Hi @LysandreJik 

1. Actually I had the intiuition something was wrong because I had this "missing weights" message again after QA training during evaluation step, and all evaluation metrics were equal to 0... like if the learned weights during QA training were not loaded at evaluation step? How to make sure eval step loads the learned weights?

2. I just retried running the command above (training + eval) from a fresh env and now I have a new issue:

```python-traceback
convert squad examples to features: 100%|█████████████████████████████████████████████████████████████████████████████████$
| 84943/84943 [52:24<00:00, 27.01it/s]
add example index and unique id: 100%|█████████████████████████████████████████████████████████████████████████████████| 8$
943/84943 [00:00<00:00, 587500.52it/s]
02/09/2020 14:42:05 - INFO - __main__ -   Saving features into cached file ./cached_train_flaubert-base-uncased_384
02/09/2020 14:44:49 - INFO - __main__ -   ***** Running training *****
02/09/2020 14:44:49 - INFO - __main__ -     Num examples = 87016
02/09/2020 14:44:49 - INFO - __main__ -     Num Epochs = 2
02/09/2020 14:44:49 - INFO - __main__ -     Instantaneous batch size per GPU = 3
02/09/2020 14:44:49 - INFO - __main__ -     Total train batch size (w. parallel, distributed & accumulation) = 3
02/09/2020 14:44:49 - INFO - __main__ -     Gradient Accumulation steps = 1
02/09/2020 14:44:49 - INFO - __main__ -     Total optimization steps = 58012
Epoch:   0%|                                                                                                               
                | 0/2 [00:00<?, ?it/sTraceback (most recent call last):                                                    
                                                  | 0/29006 [00:00<?, ?it/s]
  File "./examples/run_squad.py", line 857, in <module>
    main()
  File "./examples/run_squad.py", line 796, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "./examples/run_squad.py", line 231, in train
    outputs = model(**inputs)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/transformers/modeling_xlm.py", line 1036, in forward
    inputs_embeds=inputs_embeds,
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/transformers/modeling_flaubert.py", line 235, in forward
    tensor = tensor + self.lang_embeddings(langs)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'FlaubertModel' object has no attribute 'lang_embeddings'
Epoch:   0%|                                                                                                               
                | 0/2 [00:00<?, ?it/s]
Iteration:   0%|                                                                                                           
            | 0/29006 [00:00<?, ?it/s]
```

It seems `lang_embeddings` is not available in `FlaubertModel`: 

https://github.com/huggingface/transformers/blob/d426b58b9e32a2ffc8c8a1196143270e22054a46/src/transformers/modeling_flaubert.py#L229-L240

It is declared in XLM:

https://github.com/huggingface/transformers/blob/d426b58b9e32a2ffc8c8a1196143270e22054a46/src/transformers/modeling_xlm.py#L358-L365

Do you have any idea how to fix this? Thanks!<|||||>Hi, 

1) Indeed, you should not have gotten these warnings if the model loaded was the one that you just trained.

2) This should have been fixed with https://github.com/huggingface/transformers/commit/cfb7d108bd4ad067a03faf15255a6ea55a6c8d39, could you install from source and let me know if it fixes your issue?<|||||>@LysandreJik It's fixed now! Thank you 👍 <|||||>Great to hear! 