RobertaTokenizer different than fairseq for 'world'
`pip install fairseq`
```
roberta = torch.hub.load('pytorch/fairseq', 'roberta.base')
rt = RobertaTokenizer.from_pretrained('roberta-base')
for ex in ['Hello world', ' Hello world', ' world', 'world', 'Hello', ' Hello']:
    print(f'{ex} fairseq: {roberta.encode(ex).tolist()}, Transformers: {rt.encode(ex,  add_prefix_space=True)}')

>>>

Hello world fairseq: [0, 31414, 232, 2], Transformers: [0, 20920, 232, 2]
 Hello world fairseq: [0, 20920, 232, 2], Transformers: [0, 20920, 232, 2]
 world fairseq: [0, 232, 2], Transformers: [0, 232, 2]
world fairseq: [0, 8331, 2], Transformers: [0, 232, 2]
Hello fairseq: [0, 31414, 2], Transformers: [0, 20920, 2]
 Hello fairseq: [0, 20920, 2], Transformers: [0, 20920, 2]
```
Notice that even the token "world" is different, but results always the same with leading spaces.
Is this related to @joeddav recent work?
h/t @pnpnpn for uncovering:)
Reading more, pretty sure we only expect to have the same results as fairseq when the argument to fairseq starts with a space. Closing but would love verification/knowledge !<|||||>Yes, this comes from #2778, which changes the default behavior to automatically prepending a space when `add_special_tokens=True` for Roberta, since you want a space after `<s>`. Can be overriden with `add_prefix_space=False`. This does deviate from fairseq's encode fn, ~~but reflects the behavior of their `fill_mask` [which also prepends a space](https://github.com/pytorch/fairseq/blob/master/fairseq/models/roberta/hub_interface.py#L149).~~ Nvm, fairseq's `fill_mask` function doesn't prepend a space after all. They expect the user to know that they have to prepend a space to get correctly encoded sequences.