Confusing tokenizer result on single word
Not sure if this is expected, but it seems confusing to me: 

```python
import transformers
t=transformers.AutoTokenizer.from_pretrained('roberta-base')
t.tokenize("mystery")
```

yields two tokens, `['my', 'stery']`.

Yet

```
t.tokenize("a mystery") 
```

*also* yields two tokens, `['a', 'Ġmystery']`. I would have thought this should yield one more token than tokenizing "mystery" alone.

Hey @malmaud I think this #1196 can help you. The Roberta/GPT2 tokenizer expect a space to start. Without that, it sounds like you'll get strange behaviors.

To get the same output, in your first example, change it to 
```
t.tokenize("mystery", add_prefix_space=True)
['Ġmystery']
```<|||||>That does work, thanks. I'm still confused why this doesn't work, though:

```
t.tokenize("<s> mystery </s>")
```

gives `['<s>', 'my', 'stery', '</s>']`<|||||>Hey @malmaud, spent some time going through the source code. So like above this gives the correct result:
```
t.tokenize("mystery", add_prefix_space=True)
['Ġmystery']
```
However
```
t.tokenizer(" mystery")
['my', 'stery']
```
I thought these should be doing the same thing. In the tokenization_gpt2.py file, it says:
```
if add_prefix_space:
    text = ' ' + text
```
This should give the same results in both files then however when I add a print(text) statement before and after that I noticed I got these results. (using your example now)
```
t.tokenize("<s> mystery")
mystery
mystery
['<s>', 'my', 'stery']

t.tokenize("<s> mystery", add_prefix_space=True)
mystery
 mystery
['<s>', 'Ġmystery']
```

This means that even though we are putting a single word in with a leading space, something in the preprocessing is getting rid of the initial space(s). So we need to use the add_prefix_space=True in order to get the space back or else the function won't be using the string we are expecting it will be using.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
