[Marian] Run predictions on GPU? RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`  
Is it possible to run the Marian NMT model predictions on GPU, should not it be faster? Once I try to move the model to GPU I get the error described below. Is there any quick fix to the code to achieve this?
```
torch.__version__
Out[15]: '1.5.0'

```

The code to reproduce:
```
from transformers import MarianMTModel, MarianTokenizer
model_name = 'Helsinki-NLP/opus-mt-fr-en'
model = MarianMTModel.from_pretrained(model_name).cuda()
tokenizer = MarianTokenizer.from_pretrained(model_name)
```
Then I check the current device:
```
model.device
Out[33]: device(type='cuda', index=0)
```
Then I try to predict following the examples from documentation:
```
batch = tokenizer.prepare_translation_batch(list_of_sents)
gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3) 


  File "<ipython-input-9-54f44d46fb14>", line 1, in <module>
    gen = model.generate(**batch, num_beam=1, early_stopping = True, no_repeat_ngram_size = 3)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1146, in generate
    encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/transformers/modeling_bart.py", line 292, in forward
    inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)

  File "/home/ms/anaconda3/envs/git_face/lib/python3.7/site-packages/torch/nn/functional.py", line 1724, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select`
 
```
Is your `batch` on the correct device too?<|||||>Thank you, I moved `batch` to the GPU and it works.
`batch = tokenizer.prepare_translation_batch(sentences).to('cuda')`<|||||>More generally: `batch.to(model.device)`.