GPT2 checkpoint breaks on new transformers version (2.9.1)
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2

Language I am using the model on (English, Chinese ...): SMILES

## Description

When trying to load a gpt2 checkpoint from before 2.9.1, it loads fine. 

Checked on 2.8.0 and it works fine.

Then on 2.9.1 I can't load the model and I get this error:
```
RuntimeError: Error(s) in loading state_dict for LMWrapper:
	Missing key(s) in state_dict: "model.transformer.h.0.attn.masked_bias", "model.transformer.h.1.attn.masked_bias", "model.transformer.h.2.attn.masked_bias", "model.transformer.h.3.attn.masked_bias", "model.transformer.h.4.attn.masked_bias", "model.transformer.h.5.attn.masked_bias". 
```

LMWrapper is just a wrapper around the GPT2 Model, which is named "model" in self
     
- `transformers` version: 2.9.1
- Platform: Linux and Mac
- Python version: 3.7
- PyTorch version (GPU?): 1.4.0 GPU and CPU
- Tensorflow version (GPU?): None
- Using GPU in script?: Yes and No
- Using distributed or parallel set-up in script?: No

Do you mind giving us a reproducible code example so that we may check on our end?<|||||>I have the same issue, my trained models work on 2.8.0 but I get the same error when loading saved model on 2.9.1<|||||>Hi, sorry for not getting back to this @LysandreJik .

It seems that in Transformers 2.8.0 and below, there is no parameter called `masked_bias`.

However in 2.9+, there is now a parameter called `masked_bias` on line 109 in `modeling_gpt2.py`: https://github.com/huggingface/transformers/blob/v2.9.0/src/transformers/modeling_gpt2.py#L109

Thus, the error above is seen.

Could anything be done about this? Otherwise it renders all GPT2 checkpoints before 2.8.0 unusable, unless manually tinkering with the checkpoint to add this parameter?<|||||>Hi! I'm trying to reproduce, but I don't get an error like you do. These values indeed do not get loaded, but this does not raise a `RuntimeError`.

Do you mind giving me a short code sample so that I may debug on my side? Thanks a lot.<|||||>The weights are all proprietary unfortunately but let me try and do something.<|||||>@LysandreJik 

First, `pip install transformers==2.8.0`

Then do:
```
import torch
from transformers import GPT2Config, GPT2Model
torch.save(GPT2Model(GPT2Config(n_layer=2)), "gpt2_state_dict.pt")
```

Then, `pip install transformers==2.9.0`
and do
```
import torch
from transformers import GPT2Config, GPT2Model
gpt2 = GPT2Model(GPT2Config(n_layer=2))
state_dict = torch.load("gpt2_state_dict.pt")
gpt2.load_state_dict(state_dict)
```

And you get an error: 
```
RuntimeError: Error(s) in loading state_dict for GPT2Model:
	Missing key(s) in state_dict: "h.0.attn.masked_bias", "h.1.attn.masked_bias".
```<|||||>I see! Indeed, there's not much we can do about that, unfortunately. We have our own loading/saving method for that purpose.

Out of curiosity, why don't you use `from_pretrained`/`save_pretrained`?<|||||>I'm wrapping the model in a Pytorch Lightning module, and the most convenient way to save that is to save the state dictionary.<|||||>In that case, you could have a workaround like:

```py
import torch
from transformers import GPT2Config, GPT2Model

# Use the from_pretrained method
temp = GPT2Model.from_pretrained(directory)
temp.save_pretrained(directory)

gpt2 = GPT2Model(GPT2Config(n_layer=2))
state_dict = torch.load(f"{directory}/pytorch_model.pt")
gpt2.load_state_dict(state_dict)
```

Loading/saving using `from_pretrained` shouldn't raise an error, and once it's saved you can then load the state dict like you did. Once you have saved it using this, you can always just load the state dict.
<|||||>@LysandreJik yes, that seems like a good workaround. Thanks!<|||||>Glad I could help, sorry for taking so long to get back to you. Feel free to reopen if you face any other issues down the road.