How to use fine-tuned BERT to fill <mask>
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
I have fine-tuned a BERT model by classification task, use transformers.BertForSequenceClassification. Now,  i  want use this model to fill mask when i give a input like: 'my dog \<mask\> beautiful'. 

Can this be implemented? 
I would really appreciate if someone can teach me how to implement.

Hi @Odimmsun , `BertForSequenceClassification` is used for classification task and not for mak-filling. You can use the pre-trained BERT model as it is for make-filling. There is pipeline for this task which you can find here https://huggingface.co/transformers/usage.html#masked-language-modeling

Basically what you'll need to do is this

```python3
from transformers import pipeline

nlp = pipeline("fill-mask")
print(nlp(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks."))
```

you can also pass your own model to the pipeline using the `model` parameter.<|||||>> Hi @Odimmsun , `BertForSequenceClassification` is used for classification task and not for mak-filling. You can use the pre-trained BERT model as it is for make-filling. There is pipeline for this task which you can find here https://huggingface.co/transformers/usage.html#masked-language-modeling
> 
> Basically what you'll need to do is this
> 
> ```python
> from transformers import pipeline
> 
> nlp = pipeline("fill-mask")
> print(nlp(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks."))
> ```
> 
> you can also pass your own model to the pipeline using the `model` parameter.

Hi @patil-suraj ,thank you for your reply. Now, if i use "nlp = pipeline("fill-mask")", the model i used is not fine-tuned, but i  want to use a fine-tuned model  to  fill mask.  How should i do to implement this?

<|||||>The pre-trained BERT model does mask-filling out of the box, but if you want to use your own fine-tuned model then just pass the model(path or url) to the `model` parameter .

```python3
nlp = pipeline("fill-mask", model="your_model_path")
```<|||||>\`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-d85876ca5c10> in <module>()
      1 my_model.to('cpu')
      2 nlp = pipeline(task='fill-mask', model=my_model, tokenizer=tokenizer)
----> 3 nlp('我是你<mask>爸爸')

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
    804                 values, predictions = topk.values.numpy(), topk.indices.numpy()
    805             else:
--> 806                 masked_index = (input_ids == self.tokenizer.mask_token_id).nonzero().item()
    807                 logits = outputs[i, masked_index, :]
    808                 probs = logits.softmax(dim=0)

ValueError: only one element tensors can be converted to Python `scalars`
\`
hi, my model is fine-tuned by BertForSequenceClassification. Then i use it to fill mask, the error raised as above.  Now， i am confused.<|||||>`BertForSequenceClassification` is meant for classification, it won't work for mask-filling task. If you want to fine-tune BERT for masked lm task then you should use BertForMaskedLM<|||||>Check this example for how to fine-tune bert for masked LM

https://github.com/huggingface/transformers/tree/master/examples/language-modeling#robertabertdistilbert-and-masked-language-modeling<|||||>Thank @patil-suraj, do you know how to use BART with in-filling scheme (where spans of text are replaced with a single mask token)? I have not seen this pipline 

<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> `---------------------------------------------------------------------------
> ValueError Traceback (most recent call last)
> in ()
> 1 my_model.to('cpu')
> 2 nlp = pipeline(task='fill-mask', model=my_model, tokenizer=tokenizer)
> ----> 3 nlp('我是你爸爸')
> 
> /usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in **call**(self, *args, **kwargs)
> 804 values, predictions = topk.values.numpy(), topk.indices.numpy()
> 805 else:
> --> 806 masked_index = (input_ids == self.tokenizer.mask_token_id).nonzero().item()
> 807 logits = outputs[i, masked_index, :]
> 808 probs = logits.softmax(dim=0)
> 
> ValueError: only one element tensors can be converted to Python `scalars`
> `
> hi, my model is fine-tuned by BertForSequenceClassification. Then i use it to fill mask, the error raised as above. Now， i am confused.

老哥稳！