ValueError raises when load Flaubert from pre-train with Transformers >=3.0.0
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): **Flaubert**

Language I am using the model on (English, Chinese ...): **French**

The problem arises when using: my own modified scripts

## To reproduce

**Just load Flaubert model with from_pretrained:**
```
MODEL = "flaubert/flaubert_large_cased"
transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)
```

**Then a ValueError is raised:**

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-4235ae313fbd> in <module>()
      1     MODEL = "flaubert/flaubert_large_cased"
----> 2     transformer_layer = TFAutoModel.from_pretrained(MODEL, from_pt=True)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    421         for config_class, model_class in TF_MODEL_MAPPING.items():
    422             if isinstance(config, config_class):
--> 423                 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    424         raise ValueError(
    425             "Unrecognized configuration class {} for this kind of TFAutoModel: {}.\n"

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    479         if from_pt:
    480             # Load from a PyTorch checkpoint
--> 481             return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
    482 
    483         model(model.dummy_inputs, training=False)  # build the network with dummy inputs

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_checkpoint_in_tf2_model(tf_model, pytorch_checkpoint_path, tf_inputs, allow_missing_keys)
     91 
     92     return load_pytorch_weights_in_tf2_model(
---> 93         tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
     94     )
     95 

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    123 
    124     if tf_inputs is not None:
--> 125         tf_model(tf_inputs, training=False)  # Make sure model is built
    126 
    127     # Adapt state dict - TODO remove this and update the AWS weights files instead

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, **kwargs)
    635             heads.
    636         """
--> 637         outputs = self.transformer(inputs, **kwargs)
    638         return outputs
    639 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_flaubert.py in call(self, inputs, attention_mask, langs, token_type_ids, position_ids, lengths, cache, head_mask, inputs_embeds, training, output_attentions, output_hidden_states)
    267                 tensor_normalized = self.layer_norm1[i](tensor)
    268                 attn_outputs = self.attentions[i](
--> 269                     [tensor_normalized, attn_mask, None, cache, head_mask[i]], training=training
    270                 )
    271                 attn = attn_outputs[0]

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_xlm.py in call(self, inputs, training)
    139         Self-attention (if kv is None) or attention over source sentence (provided by kv).
    140         """
--> 141         input, mask, kv, cache, head_mask, output_attentions = inputs
    142         # Input is (bs, qlen, dim)
    143         # Mask is (bs, klen) (non-causal) or (bs, klen, klen)

ValueError: not enough values to unpack (expected 6, got 5)
```
## Expected behavior
Load the model successfully. 

- It can be achieved with Transformers version <= 2.11.0

- I also try CamemBERT, and Bert-like models in other languages like tr, ru, es, pt, it. None of them have this problem.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: **>=3.0.0 ( 3.0.0, 3.0.1, 3.0.2 all have this problem.)**

(I try this on google colab with GPU and TPU. The environment does not matter here. But I still provide the one with GPU.)
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

Hi @wzmJimmy,
Did you get any solution for this problem? I am also facing the same problem. Please let me know if you are able to make it run. Thanks.<|||||>Well, my solution is inside the expected behavior part. Downgrade transformers to version lower than 3.0.0 and it just works. I post the bug report just to ask what's wrong and how to fix it in the newer version.