GPT2 is not jit-traceable
## Information

I would like to use Pytorch tracing on a pretrained GPT2 model, but I run into these warnings for the attention layers:
```
python3.8/site-packages/transformers/modeling_gpt2.py:164: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  w = w / (float(v.size(-1)) ** 0.5)
python3.8/site-packages/transformers/modeling_gpt2.py:169: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  mask = self.bias[:, :, ns - nd : ns, :ns]
```
The first warning concern the same line as the one reported in #3954 (and fixed by #3955).

## To reproduce

You can run the following:
```
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')
model.eval()

tokens=tokenizer('The cat is on the table.', return_tensors='pt')['input_ids']

with torch.jit.optimized_execution(True):
    traced_model = torch.jit.trace(model, tokens)
```

## Environment info
     
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0+cpu (False)
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO


Hi! I believe those are warnings and not errors? Does it change the expected results when tracing the model?<|||||>You are right, despite the warnings (and my - limited - understanding of what should work inside tracing), the output of the compiled model with different inputs are comparable to the base ones. 

Thanks a lot!<|||||>Glad it works!<|||||>@gcompagnoni @LysandreJik 
Hi, there
Why do I try the example below occurs error ? issue reported in #15598

```
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')
model.eval()

tokens=tokenizer('The cat is on the table.', return_tensors='pt')['input_ids']

with torch.jit.optimized_execution(True):
    traced_model = torch.jit.trace(model, tokens)
```
