Predict masked word at the beginning of the sentence
Case 1: I have a sentence "After the campaign finance laws changed, Albert ran for mayor of his city."  which I have modified as
\<s\> \<mask\>  the campaign finance laws changed, he ran for mayor of his city. \</s\>.
This input is passed through pre-trained RobertaBase and RobertaLarge (RobertaTokenizer, RobertaForMaskedLM).
The predicted top-10 words are:
['When', 'After', 'Before', 'Once', 'As', 'Until', 'While', 'With', 'Since', 'Then']
Case 2: But when we have masked word in the middle of the sentence:
\<s\> He ran for mayor of his city <mask> the campaign finance laws changed. \</s\>
The predicted top-10 words are:
['Ġbefore', 'Ġafter', 'Ġwhen', 'Ġuntil', 'Ġas', 'Ġand', 'Ġbut', 'Ġonce', 'Ġbecause', ',']

Why the special character is added in the 2nd case and not case 1?


The special characters mean that these words are preceded by a space. This is not the case when the words are the start of sentences!<|||||>So, if I insert a space at the beginning of the sentence before [MASK], I should words beginning with special character or the tokenizer removes those extra spaces at the beginning?<|||||>Yes, the encoding of the first word will change if you add a space. The tokeniser does not normalise whitespace. Try adding 2 spaces, a tab or a newline character. Please read about `add_prefix_space=True` and `is_split_into_words=True` on https://huggingface.co/transformers/model_doc/roberta.html#robertatokenizer<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.