pipeline fill_mask.py - needs to convert input_ids to cpu before calling numpy
https://github.com/huggingface/transformers/blob/b7d264be0d70d5309dbf24a6c7ebba1e073bdda5/src/transformers/pipelines/fill_mask.py#L127

Should be 

tokens = input_ids.cpu().numpy()

```

from transformers import AutoTokenizer, AutoModel, AutoModelForMaskedLM, pipeline
#tokenizer = AutoTokenizer.from_pretrained("google/fnet-base")
#model = AutoModelForMaskedLM.from_pretrained("google/fnet-base").cuda()
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForMaskedLM.from_pretrained("bert-base-uncased").cuda().half()
unmasker = pipeline('fill-mask', model=model, tokenizer=tokenizer, device=0)
unmasker("Hello I'm a [MASK] model.")

[
    {"sequence": "hello i'm a new model.", "score": 0.12073223292827606, "token": 351, "token_str": "new"},
    {"sequence": "hello i'm a first model.", "score": 0.08501081168651581, "token": 478, "token_str": "first"},
    {"sequence": "hello i'm a next model.", "score": 0.060546260327100754, "token": 1037, "token_str": "next"},
    {"sequence": "hello i'm a last model.", "score": 0.038265593349933624, "token": 813, "token_str": "last"},
    {"sequence": "hello i'm a sister model.", "score": 0.033868927508592606, "token": 6232, "token_str": "sister"},
]
```
Will produce the right result with the change. But will raise an error without the change:

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
Indeed! Would you like to open a PR to fix this issue?<|||||>Maybe after I finish finding some more bugs... I'm looking at fnet and found another bug. I'm tryin to get fnet working in half mode...
<|||||>Exactly same issue. 

https://colab.research.google.com/drive/1F7XxC1tTVCALBS6X8DEtEPZwXvznuLs7?usp=sharing

Reproducable at Colab.

Don't know just that line is issue, or more big issues are coming...
I'm looking on it too!

<|||||>Hi,

This was normally fixed on master (well the exact proposed example does not work because the model is `half()` and `softmax` is not implemented for `f16` in PyTorch (you can either use `f32` or override the `_forward` method to cast things back to `f32` after it went through the model.

Does that help ?

<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>This was fixed on master, feel free to reopen.