strange output of fast/slow tokenizers
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.6.6
- PyTorch version (GPU?): 1.5.0+cpu (False)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help


 tokenizers: @mfuntowicz
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
@thomwolf 
@Narsil 
 -->

## Information

First of all, this might be a problem of the fast tokenizer, ,but I am not 100% sure, because the bug occurs when I use `AutoTokenizer`, for which the code is in `transformers`. I didn't check the usage directly from the tokenizer's object.

Second, the problematic string is not meaningful. I am glad to see the fast tokenizer is available for XLM-Roberta, and I prefer to make sure it works as expected. So I compared the results from the slow/fast tokenizers in order to make sure the results are the same.

However, with this string (without the leading/training single quote)

    '=LLC-s nmrcsss mtiaiol!@"ccc technooay @"ccc"@"ccc'

The result is different, you can see the output below.

Fast tokenizer
```
{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 238, 10060, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
```

Slow tokenizer
```
{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 10060, 238, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
```

The encoding at the end are 

`... 58, 238, 10060, 2` and `... 58, 10060, 238, 2` respectively.


Furthermore, if I remove any character from it, the results become the same of these 2 tokenizers.



## To reproduce

Steps to reproduce the behavior:

```
from transformers import AutoTokenizer

tokenizer_fast = AutoTokenizer.from_pretrained("xlm-roberta-large", use_fast=True)
tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-large", use_fast=False)

# This string cause problem.
s = '=LLC-s nmrcsss mtiaiol!@"ccc technooay @"ccc"@"ccc'
o1 = tokenizer_fast.batch_encode_plus([s])
o2 = tokenizer.batch_encode_plus([s])
if not o1 == o2:
    print('output are different!')
    print(f'string: {s}')
    print(o1)
    print(o2)

# check substring
s2 = s
m = 0
while True:

    m += 1
    if m > 100000:
        break

    n = len(s2)
    for i in range(0, n):

        # substring of one char removed
        s_temp = s2[0:i] + s2[i+1:]
        o1 = tokenizer_fast.batch_encode_plus([s_temp])
        o2 = tokenizer.batch_encode_plus([s_temp])

        if not o1 == o2:
            print(s_temp)
            print('-------------------')
            s2 = s_temp
            break

    if len(s2) == n:
        print('no substring with 1 char fewer cause problem, stopped.')
        break
```

Output:

```
output are different!
string: =LLC-s nmrcsss mtiaiol!@"ccc technooay @"ccc"@"ccc
{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 238, 10060, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 10060, 238, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
no substring with 1 char fewer cause problem, stopped.

Process finished with exit code 0
```

## Expected behavior

The results are expected to be the same.

@chiapas , We will review this issue and propose code changes soon.<|||||>@chiapas , I executed the same code in GoogleColab. I am getting the same tokens for both of them. Here is the output.


`{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 238, 10060, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
{'input_ids': [[0, 2203, 23708, 441, 9, 7, 653, 39, 19437, 7, 7, 7, 347, 4526, 34837, 38, 981, 58, 238, 10060, 128500, 31, 3337, 1374, 58, 238, 10060, 58, 981, 58, 238, 10060, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
no substring with 1 char fewer cause problem, stopped.`

Can you please give me more details about the issue ?<|||||>Looks like this issue doesn't occurs if I upgrade my python version to `3.6.9` from `3.6.6`. I am not sure why there is a problem when this code sample is executed with python `3.6.6`, but since it is quite old version, I won't ask further investigation, and this issue could be closed.