resize_token_embeddings doesn't work as expected for BertForMaskedLM
## 🐛 Bug

<!-- Important information -->

Model I am using : **BertForMaskedLM**

Language I am using the model on (English, Chinese....): **multilingual**

The problem arise when using:
* [x] the official example scripts: **BertForMaskedLM**

The tasks I am working on is:
* [x] my own task or dataset: **Finetuning with newly added tokens**.

## To Reproduce

Steps to reproduce the behavior:

```python
# Get model
model = BertForMaskedLM.from_pretrained("bert-base-multilingual-cased")

# define input
input = torch.tensor([[1,2,3,4,5]])

# before experiment
print(f"BERT num embeddings before\t: {model.bert.embeddings.word_embeddings.num_embeddings}")
print(f"LM Decoder num embedding before\t: {model.cls.predictions.decoder.out_features}")
print(f"MLM Loss\t\t\t: {model(input_ids=input, masked_lm_labels=input)[0]}")

# change embedding size for experiment
model.resize_token_embeddings(119547 + 5)
print(f"\nBERT num embeddings after\t: {model.bert.embeddings.word_embeddings.num_embeddings}")
print(f"LM Decoder num embedding after\t: {model.cls.predictions.decoder.out_features}")
# print failuire
try:
  print(f"MLM Loss\t\t\t: {model(input_ids=input, masked_lm_labels=input)[0]}")
except RuntimeError:
  print(" ---- Forward Pass failed ---- ")
```

This is the output that I see
```
BERT num embeddings before	: 119547
LM Decoder num embedding before	: 119547
MLM Loss			: 18.269268035888672

BERT num embeddings after	: 119552
LM Decoder num embedding after	: 119547
 ---- Forward Pass failed ---- 
```

If you see the LM decoder did not change the embedding size, hence won't ever predict the new tokens.

The expectation would be that `resize_token_embeddings` handles the FC inside LM decoder.

I tried also using BertConfig 

```python
config = BertConfig.from_pretrained("bert-base-multilingual-cased")
config.vocab_size = 119547 + 5
model = BertForMaskedLM.from_pretrained("bert-base-multilingual-cased", config=config)
```

Which naturally results in error while loading `state_dict`

```
RuntimeError: Error(s) in loading state_dict for BertForMaskedLM:
	size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([119547, 768]) from checkpoint, the shape in current model is torch.Size([119554, 768]).
	size mismatch for cls.predictions.bias: copying a param with shape torch.Size([119547]) from checkpoint, the shape in current model is torch.Size([119554]).
	size mismatch for cls.predictions.decoder.weight: copying a param with shape torch.Size([119547, 768]) from checkpoint, the shape in current model is torch.Size([119554, 768]).
```
Hi! May I know what version are you using? On the current master branch, this is the result:

```py
import torch
from transformers import BertForMaskedLM

input_ids = torch.tensor([[1,2,3,4,5]])
print(model.bert.embeddings.word_embeddings.num_embeddings)  # 119547
print(model.cls.predictions.decoder.out_features)  # 119547
model.resize_token_embeddings(119547 + 5)
print(model.bert.embeddings.word_embeddings.num_embeddings)  # 119552
print(model.cls.predictions.decoder.out_features)  # 119552
```<|||||>Hmm, I am also on 2.1.1. Looks like in your example you're not loading the model from a pretrained model. (Though I am not sure how that makes a difference, but looks like it does)


```python
model = BertForMaskedLM.from_pretrained(model_type, cache_dir="/tmp/bert_models")

print(transformers.__version__) #2.1.1
print(model.bert.embeddings.word_embeddings.num_embeddings)  # 119547
print(model.cls.predictions.decoder.out_features)  # 119547
model.resize_token_embeddings(119547 + 5)
print(model.bert.embeddings.word_embeddings.num_embeddings)  # 119552
print(model.cls.predictions.decoder.out_features)  # 119547
```

And as per the current master also I don't see where we copy over the params from the old `cls` to the new updated `cls` with the new number of tokens as we do it for the embedding layer [here](https://github.com/huggingface/transformers/blob/de890ae67d43e1e5d031a815dab5dfed081e9a95/transformers/modeling_utils.py#L196).


In the next comment, I can share the temporary hack that I have.<|||||>```python
class CustomBertForMaskedLM(BertForMaskedLM):
    def __init__(self, config):
        super().__init__(config)

    def resize_embedding_and_fc(self, new_num_tokens):
      # Change the FC 
      old_fc = self.cls.predictions.decoder
      self.cls.predictions.decoder = self._get_resized_fc(old_fc, new_num_tokens)
      
      # Change the bias
      old_bias = self.cls.predictions.bias
      self.cls.predictions.bias = self._get_resized_bias(old_bias, new_num_tokens)
      
      # Change the embedding
      self.resize_token_embeddings(new_num_tokens)
    
    
    def _get_resized_bias(self, old_bias, new_num_tokens):
        old_num_tokens = old_bias.data.size()[0]
        if old_num_tokens == new_num_tokens:
            return old_bias

        # Create new biases
        new_bias = nn.Parameter(torch.zeros(new_num_tokens))
        new_bias.to(old_bias.device)

        # Copy from the previous weights
        num_tokens_to_copy = min(old_num_tokens, new_num_tokens)
        new_bias.data[:num_tokens_to_copy] = old_bias.data[:num_tokens_to_copy]
        return new_bias
    
    def _get_resized_fc(self, old_fc, new_num_tokens):

        old_num_tokens, old_embedding_dim = old_fc.weight.size()
        if old_num_tokens == new_num_tokens:
            return old_fc

        # Create new weights
        new_fc = nn.Linear(in_features=old_embedding_dim, out_features=new_num_tokens)
        new_fc.to(old_fc.weight.device)

        # initialize all weights (in particular added tokens)
        self._init_weights(new_fc)

        # Copy from the previous weights
        num_tokens_to_copy = min(old_num_tokens, new_num_tokens)
        new_fc.weight.data[:num_tokens_to_copy, :] = old_fc.weight.data[:num_tokens_to_copy, :]
        return new_fc
```


Using this, I can achieve the output as expected.

![image](https://user-images.githubusercontent.com/7589415/68256992-a3f7d580-ffe6-11e9-9496-5624152b85c2.png)
<|||||>Indeed I had forgotten the line about the model initialization, but it is initialized from a pretrained checkpoint as you have done. Running your code gives me the same results:

```py
from transformers import BertForMaskedLM
import torch

model = BertForMaskedLM.from_pretrained("bert-base-cased")

print(model.bert.embeddings.word_embeddings.num_embeddings)  # 28996
print(model.cls.predictions.decoder.out_features)  # 28996
model.resize_token_embeddings(119547 + 5)
print(model.bert.embeddings.word_embeddings.num_embeddings)  # 119552
print(model.cls.predictions.decoder.out_features)  # 119552
```

 but please note I am running on the **master branch**, not on the official pypi release. In the official release the predictions layer indeed isn't correctly tied, but it has been fixed since then with #1721.

You can install from master with: `pip install git+https://github.com/huggingface/transformers`.

As for where the tying happens, it is [in the superclass `PreTrainedModel`](https://github.com/huggingface/transformers/blob/master/transformers/modeling_utils.py#L114) which ties the input embeddings and output embeddings if there are any.<|||||>@LysandreJik This issue is only partially fixed. In _tie_or_clone_weights(), it tries to fix output_embeddings.bias, but output_embeddings is actually self.cls.predictions.decoder, which has no bias term. Instead, self.cls.predictions has a bias term. This bias term is not fixed, and exceptions will occur. 

I'll have to use @praateekmahajan 's hack temporarily 😃 <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This is still an issue, I'm reopening it.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@LysandreJik do you know if its `wontfix` or just in your backlog. Lmk if I can help somehow. <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
