downstream tasks requiring bi-directional context, such as SQuAD (question answering, see Lewis, Lui, Goyal et al., part 4.2). If you would like to fine-tune a model on a masked language modeling
task, you may leverage the run_mlm.py script.
Here is an example of using pipelines to replace a mask from a sequence:
```py
from transformers import pipeline
unmasker = pipeline("fill-mask")
```
This outputs the sequences with the mask filled, the confidence score, and the token id in the tokenizer vocabulary:
```py
from pprint import pprint
pprint(
...     unmasker(
...         f"HuggingFace is creating a {unmasker.tokenizer.mask_token} that the community uses to solve NLP tasks."
...     )
... )
[{'score': 0.1793,
  'sequence': 'HuggingFace is creating a tool that the community uses to solve '
              'NLP tasks.',
