Loading Basic GPT-2  model gives warning that attention layers weren't loaded from pre-trained weights
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.0
- Platform: Linux-4.19.0-16-cloud-amd64-x86_64-with-debian-10.9
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.4.1 (True)
- Using GPU in script?: Yes. One GPU on Google Cloud Compute
- Using distributed or parallel set-up in script?: No

Tagging the following people for assistance
- gpt2: @patrickvonplaten, @LysandreJik


## Information

I'm using the GPT-2 vanilla model. Locally on my MacbookPro, my code runs as expected. When I try to run my Jupyter notebook on GCP, I however encounter an unseen warning. I provided the env details of said GCP instance above. I'm simply trying to load the vanilla GPT-2 model, but I keep getting the warning that the attention layers are not being initialized from the pre-trained weights as intended.

I get the following warning message:
`Some weights of GPT2Model were not initialized from the model checkpoint at gpt2 and are newly initialized: ['h.5.attn.masked_bias', 'h.6.attn.masked_bias', 'h.1.attn.masked_bias', 'h.8.attn.masked_bias', 'h.2.attn.masked_bias', 'h.10.attn.masked_bias', 'h.7.attn.masked_bias', 'h.4.attn.masked_bias', 'h.11.attn.masked_bias', 'h.9.attn.masked_bias', 'h.0.attn.masked_bias', 'h.3.attn.masked_bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.`

This happens when I attempt to execute the super simple model loading statement:

` pretrained_transformer = GPT2Model.from_pretrained('gpt2')`

I've seen similar issues people have posted, however, they tend to be loading a model and trying to load the weights to a different type of model. Here I am strictly trying to load a GPT2 model weight set and then configure the same GPT2 model with said weights.

I'm worried these warnings are real and it seems my experiments on GCP are not looking the same as the ones locally due to the weights not being loaded properly.

## To reproduce

Steps to reproduce the behavior:

1. Make GCP notebook
2. Try to load GPT-2 model

## Expected behavior

The behaviour I desire is no warning messages and being able to use all the fully trained pre-trained weights as I'm able to do locally. 


These warnings mention that buffers are not loaded, which is normal - they're created during the model initialization.

There was an attribute missing on the `GPT2Model` which led the warnings to still be raised, I'm fixing this in #11709!<|||||>Installing from source should resolve the warnings issue :) 