Why does the T5Tokenizer prepend and '_' to every token?
I am using the T5Tokenizer as follows:

```
In [119]: tokenizer = T5Tokenizer.from_pretrained('t5-small')

In [120]: input_ids = tokenizer.encode_plus('I love my dog', return_tensors='pt')

In [121]: [tokenizer.convert_ids_to_tokens([ele]) for ele in input_ids['input_ids'][0]]
Out[121]: [['▁I'], ['▁love'], ['▁my'], ['▁dog']]
```
I even tried another one as follows:

```
In [129]: [tokenizer.convert_ids_to_tokens([ele]) for ele in input_ids['input_ids'][0]]
Out[129]: [['▁I'], ['▁love'], ['▁my'], ['▁school'], ['▁National'], ['x'], ['y'], ['z']]
```
Above, shouldn't 'x', 'y', and 'z' have # prepended to them  as they are part of the same word?

Why do I see an underscore before every token? Is this related to how it is sent inside T5?
Hi, the T5 tokenizer is a SentencePiece tokenizer, and that's the way SentencePiece works. This underscore means that it's the start of a word. When it's not the start of a word, it's not prepended by anything.

I think you're thinking of the # symbol because you're used to the BertTokenizer (wordpiece). SentencePiece works a bit differently!