Can BertForMaskedLM be used to predict out-of-vocabulary words?
Hi,

I have this text:

```
[CLS] This is a picture of a boa.
```

And would like to have the predictions of the `BertForMaskedLM` model for the word `boa`, without masking this word.

However, when I tokenize the text to give it to the network, I get:

```
['[CLS]', 'This', 'is', 'a', 'picture', 'of', 'a', 'b', '##oa', '.']
```

And the network gives me predictions for `b` and `##oa`. But nothing relevant. Could I get predictions for `boa`?


What do yuu get when you multiply the probabilities for words in these 2 places. Probability for b times probability for "##oa".
['[CLS]', 'This', 'is', 'a', 'picture', 'of', 'a', '[MASK]', '[MASK]', '.']
['[CLS]', 'This', 'is', 'a', 'picture', 'of', 'a', '[MASK]', '##oa', '.']

But there is also whole word masking model is realized by Google team, I hope it will be added here soon. It mask all the parts of these words at the same time, so it would give a better accuracy (1% for some tasks).
https://github.com/google-research/bert

<|||||>Nothing interesting.

Moreover, I would not like to mask words. I just want to get predictions with the word in clear.<|||||>Well, another solution is to recreate bert with a large vocabulary. This would take days on TPU.
<|||||>Okay thanks for the answer<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
