Forward pass with GPT2 using both past and attention_mask as an input leads to dimension error
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ x ] my own task or dataset: (give details below)

## To reproduce run the following code:

```
from transformers import GPT2LMHeadModel
import torch

model = GPT2LMHeadModel.from_pretrained('gpt2')

input_ids = torch.tensor([8, 8, 0, 50256, 50256]).unsqueeze(0)
attn_mask = torch.tensor([1, 1, 1, 0, 0]).unsqueeze(0)

# first step there is no past so lets get it from the model and append new embedding id to inputs and extend the attn_mask
logits_output, past = model(input_ids, attention_mask=attn_mask)
next_token = torch.argmax(logits_output[:, -1, :]).unsqueeze(0)
input_ids = torch.cat([input_ids, next_token.unsqueeze(-1)], dim=-1) 
attn_mask = torch.cat([attn_mask, torch.ones((attn_mask.shape[0], 1)).long()], dim=1)


# now we have a past so we can use it to speed up training
model_inputs = model.prepare_inputs_for_generation(input_ids=input_ids, past=past)
logits_output, past = model(**model_inputs, attention_mask=attn_mask)  # this leads to an error which it should not
```

## Expected behavior

No error should be thrown and the forward pass to get `logits_output` and `past` should work

## Environment info
     
- `transformers` version: 2.5.1
- Platform: Linux-5.3.0-40-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


FYI - I created a stackoverflow question about this here: https://stackoverflow.com/questions/60459292/using-past-and-attention-mask-at-the-same-time-for-gpt2<|||||>@Damiox - answered you stackoverflow question :-)