Error when trying to reuse hidden states in CTRL
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): CTRL

Language I am using the model on (English, Chinese....): English

The problem arise when using:

My own script, the colab link is available [here](https://colab.research.google.com/drive/143T4sBda4r2nDYzmuNwi-ZFTbhJWfeOW)

The stack trace is:

```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-9-ac2d93f8c410> in <module>()
      3   for i in range(3):
      4     print(i)
----> 5     logits, past = model(**inputs, past=past)
      6     logits = logits[0, -1]
      7 

8 frames
/usr/local/lib/python3.6/dist-packages/transformers/modeling_ctrl.py in scaled_dot_product_attention(q, k, v, mask, attention_mask, head_mask)
     64 
     65     if mask is not None:
---> 66         scaled_attention_logits += (mask * -1e4)
     67 
     68     if attention_mask is not None:

RuntimeError: The size of tensor a (13) must match the size of tensor b (7) at non-singleton dimension 3
```

The tasks I am working on is:

Generating text with CTRL

## To Reproduce

Just run the colab from the link I posted above

The main part of the code is:

```
input_ids = torch.tensor(tokenizer.encode("Links Hello, my dog is cute")).unsqueeze(0).to(device)  
inputs = {'input_ids': input_ids}

with torch.no_grad():
  past = None
  for i in range(3):
    print(i)
    logits, past = model(**inputs, past=past)
    logits = logits[0, -1]
    
    next_token = logits.argmax()
    input_ids = torch.cat([input_ids, next_token.view(1, 1)], dim=1)
    
    inputs = {'input_ids': input_ids}
```

## Expected behavior

passing in `past` should not throw an error and should speed up generation

## Environment

* OS: Linux
* Python version: 3.6.8
* PyTorch version: 1.3.0+cu100
* PyTorch Transformers version (or branch): 2.1.1
* Using GPU: yes
* Distributed of parallel setup: No
* Any other relevant information: I'm running on an extended memory colab instance with a K80

The same error occurs with the library installed with `git clone` (_master_ version) + torch v1.3.0 + python v3.6.8.
[Here](https://colab.research.google.com/drive/1nawWX6Lrfh9ZVKyRfTLgFSIG355xkPRy#scrollTo=n93UZjq5EIE_) is a more verbose version of the Colab Notebook posted by @bkkaggle with Google Colab.<|||||>I've just pushed a fix on the branch `fix-ctrl-past`. It should be in the next release.<|||||>Thanks, closing