BART -- RuntimeError: expected device cuda:0 but got device cpu
# 🐛 Bug
@sshleifer

I'm using BART model (bart-large), when I try to use the BartForMaskedLM i'm getting the above error. The reason is that in the _combine_masks (line 146 in modeling_bart) is creating a tensor without the device. so by default it is on CPU.  To reproduce - simply use BartForMaskedLM model with GPU. 
can you help? am I missing anything? 

Additional details: 
----------------------
- `transformers` version: 2.5.1
- Python version: 3.7.4
- PyTorch version (GPU?): 1.3.0 (with GPU)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no 



stuck trace:
File "/specific/netapp5_2/gamir/adi/git/BERTese/lama/training.py", line 151, in train_and_eval
    outputs = model(b_in_tensor, lm_labels=b_label_tensor)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/transformers/modeling_bart.py", line 925, in forward
    decoder_cached_states=decoder_cached_states,
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/transformers/modeling_bart.py", line 844, in forward
    decoder_cached_states=decoder_cached_states,
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/transformers/modeling_bart.py", line 499, in forward
    need_attn_weights=self.output_attentions,
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/transformers/modeling_bart.py", line 372, in forward
    attn_mask=attention_mask,
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/trans_py37/lib/python3.7/site-packages/transformers/modeling_bart.py", line 629, in forward
    attn_weights = attn_weights.view(bsz, self.num_heads, tgt_len, src_len) + attn_mask
RuntimeError: expected device cuda:0 but got device cpu

Thanks,
Adi.
Can you reproduce with the code on master?

```
git clone https://github.com/huggingface/transformers
cd transformers
pip install .
```
(duplicate of https://github.com/huggingface/transformers/issues/3079)
<|||||>Closing for now, reopen if this is broken on the latest code. Otherwise, it will be in the next pip release. Thanks!