tokenizer.convert_ids_to_tokens(tokenizer.convert_tokens_to_ids(x)) returning a different result
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

I have a trained Esperanto Tokenizer, the training code was taken from the how_to_train example. I'm using Roberta Tokenizer as RobertaTokenizerFast doesn't work with trainer.py yet (or last time I checked).

```python
from transformers import RobertaTokenizer

alt_tokenizer = RobertaTokenizer.from_pretrained("./drive/My Drive/EsperBERTo", max_len=512)

alt_tokenizer.convert_ids_to_tokens(alt_tokenizer.convert_tokens_to_ids(alt_tokenizer.tokenize('Neniu atendas la Hispanan Inkvizicion.')))

```
output is: ['Neniu', 'Ġatendas', 'Ġla', 'ĠHispan', 'an', 'ĠIn', 'kvizi', 'cion', '.']

## Expected behavior

the same output as input

## Environment info
tokenizers               0.8.0rc1       
transformers             2.11.0   
     
- `transformers` version: 2.11.0
- tokenizers version: 0.8.0rc1
- Platform: colab
- Python version: 3.6
- Using GPU in script?: No
