XLMRobertaTokenizerFast gives incorrect offset mappings when loaded from disk
## Environment info

- `transformers` version: 4.5.0
- Platform: Windows 10 (replicated on Ubuntu 18.04 as well)
- Python version: 3.6.8
- PyTorch version (GPU?): n/a
- Tensorflow version (GPU?): n/a
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

Full package list:
```
certifi==2020.12.5
chardet==4.0.0
click==7.1.2
dataclasses==0.8
filelock==3.0.12
idna==2.10
importlib-metadata==3.10.0
joblib==1.0.1
numpy==1.19.5
packaging==20.9
protobuf==3.15.7
pyparsing==2.4.7
regex==2021.4.4
requests==2.25.1
sacremoses==0.0.44
sentencepiece==0.1.95
six==1.15.0
tokenizers==0.10.2
tqdm==4.60.0
transformers==4.5.0
typing-extensions==3.7.4.3
urllib3==1.26.4
zipp==3.4.1
```

### Who can help
@LysandreJik 

## Information

Model I am using (Bert, XLNet ...): n/a

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

```python
>>> tokenizer = XLMRobertaTokenizerFast.from_pretrained('xlm-roberta-base')
>>> tokenizer.save_pretrained('test')
('test\\tokenizer_config.json', 'test\\special_tokens_map.json', 'test\\sentencepiece.bpe.model', 'test\\added_tokens.json')
>>> tokenizer_loaded = XLMRobertaTokenizerFast.from_pretrained('test')
>>> tokenizer('hello world', return_offsets_mapping=True)
{'input_ids': [0, 33600, 31, 8999, 2], 'attention_mask': [1, 1, 1, 1, 1], 'offset_mapping': [(0, 0), (0, 4), (4, 5), (6, 11), (0, 0)]}
>>> tokenizer_loaded('hello world', return_offsets_mapping=True)
{'input_ids': [0, 33600, 31, 8999, 2], 'attention_mask': [1, 1, 1, 1, 1], 'offset_mapping': [(0, 0), (0, 4), (4, 5), (5, 11), (0, 0)]}
```

## Expected behavior

The loaded tokenizer should produce the same offset mapping as the original tokenizer. It seems like the tokenizer loaded from disk does not account for spaces in its offset mapping.

This does not appear to occur with `BertTokenizerFast`, `GPT2TokenizerFast`, or `RobertaTokenizerFast`.<|||||>This bug was introduced in `4.3.0`, I do not encounter this issue when running with `transformers==4.2.2` <|||||>Hi @johnmccain! This is unfortunately a known issue, and an issue on which we're working. It's a complex issue as it requires updating the `tokenizer.json` on the hub, which needs to be done very carefully in order to not break backwards compatibility for users.

However, I would argue that it's the tokenizer loaded directly from the hub that has wrong offsets. If we look at the following:
```py
from transformers import XLMRobertaTokenizerFast

checkpoint = "xlm-roberta-base"

tokenizer = XLMRobertaTokenizerFast.from_pretrained(checkpoint)
tokenizer.save_pretrained(checkpoint + "_local")
tokenizer_loaded = XLMRobertaTokenizerFast.from_pretrained(checkpoint + "_local")

print(tokenizer('hello world son', return_offsets_mapping=True))
print(tokenizer('hello world son').encodings[0].tokens)
print(tokenizer_loaded('hello world son', return_offsets_mapping=True))
```
We'll obtain the following:
```
{'input_ids': [0, 33600, 31, 8999, 775, 2], 'attention_mask': [1, 1, 1, 1, 1, 1], 'offset_mapping': [(0, 0), (0, 4), (4, 5), (6, 11), (12, 15), (0, 0)]}
['<s>', '▁hell', 'o', '▁world', '▁son', '</s>']
{'input_ids': [0, 33600, 31, 8999, 775, 2], 'attention_mask': [1, 1, 1, 1, 1, 1], 'offset_mapping': [(0, 0), (0, 4), (4, 5), (5, 11), (11, 15), (0, 0)]}
```

I would argue that these (the second example) are the true offsets:
```
[ '<s>',  '▁hell',    'o', '▁world',   '▁son',  '</s>']
[(0, 0),  (0, 4),  (4, 5),  (5, 11),  (11, 15),  (0, 0)]
```

---

### Solving this

We are aware of this issue and are working on solving it. You should have been getting the wrong offsets in version v4.2.2, because this was an offset bug fixed in a recent `tokenizers` version; hence why the tokenizer file on the hub gives you the wrong offsets, while the conversion using the recent `tokenizers` yields accurate offsets.

 In the meantime, you can work around this by forcing your `XLMRobertaTokenizerFast` to proceed to a conversion from the slow tokenizer, which will correctly instantiate your fast tokenizer. You can do so with the `from_slow` flag:

```
tokenizer = XLMRobertaTokenizerFast.from_pretrained("xlm-roberta-base", from_slow=True)
```

Please let me know if this fixes your issue.<|||||>Hey @LysandreJik, thanks for the quick reply! Glad to know y'all are already aware of this and working on a solution.

I'm not sure that I agree that the tokenizer from the hub is wrong here. It seems to me that the difference between the offsets from the tokenizer loaded via the hub vs. the tokenizer loaded from disk is the inclusion of leading spaces in the offset mappings. My understanding of the offset mappings is that they indicate the indices in the original string that the token represents, excluding the whitespace preceding the token. 

Tokenizers other than `XLMRobertaTokenizerFast` are consistent with that behavior and do not include leading whitespace in their token offsets (whether loaded from the hub or from disk).

Example:

```python
from transformers import BertTokenizerFast, XLMRobertaTokenizerFast, RobertaTokenizerFast
xlm_roberta_tokenizer = XLMRobertaTokenizerFast.from_pretrained('xlm-roberta-base')
xlm_roberta_tokenizer.save_pretrained('test')
xlm_roberta_tokenizer_loaded = XLMRobertaTokenizerFast.from_pretrained('test')
roberta_tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base')
bert_tokenizer = BertTokenizerFast.from_pretrained('bert-base-uncased')

test_str = 'hello world son'

print('\n', 'XLMRobertaTokenizerFast from hub')
tokenized = xlm_roberta_tokenizer(test_str, return_offsets_mapping=True)
for (offset_start, offset_end), token in zip(tokenized.offset_mapping, tokenized.encodings[0].tokens):
    print(repr(token), repr(test_str[offset_start:offset_end]))
    
print('\n', 'XLMRobertaTokenizerFast from disk')
tokenized = xlm_roberta_tokenizer_loaded(test_str, return_offsets_mapping=True)
for (offset_start, offset_end), token in zip(tokenized.offset_mapping, tokenized.encodings[0].tokens):
    print(repr(token), repr(test_str[offset_start:offset_end]))
    
print('\n', 'RobertaTokenizerFast from hub')
tokenized = roberta_tokenizer(test_str, return_offsets_mapping=True)
for (offset_start, offset_end), token in zip(tokenized.offset_mapping, tokenized.encodings[0].tokens):
    print(repr(token), repr(test_str[offset_start:offset_end]))

print('\n', 'BertTokenizerFast from hub')
tokenized = bert_tokenizer(test_str, return_offsets_mapping=True)
for (offset_start, offset_end), token in zip(tokenized.offset_mapping, tokenized.encodings[0].tokens):
    print(repr(token), repr(test_str[offset_start:offset_end]))
```

Output:
```
 XLMRobertaTokenizerFast from hub
'<s>' ''
'▁hell' 'hell'
'o' 'o'
'▁world' 'world'
'▁son' 'son'
'</s>' ''

 XLMRobertaTokenizerFast from disk
'<s>' ''
'▁hell' 'hell'
'o' 'o'
'▁world' ' world'
'▁son' ' son'
'</s>' ''

 RobertaTokenizerFast from hub
'<s>' ''
'hello' 'hello'
'Ġworld' 'world'
'Ġson' 'son'
'</s>' ''

 BertTokenizerFast from hub
'[CLS]' ''
'hello' 'hello'
'world' 'world'
'son' 'son'
'[SEP]' ''
```
<|||||>> My understanding of the offset mappings is that they indicate the indices in the original string that the token represents, excluding the whitespace preceding the token.

The offset mapping indicates the indices in the original string that the token represents. If the token contains any leading whitespace, then by default it will be part of the offset mapping.

The Roberta tokenizer is an exception as it uses an option available only for the byte-level tokenizers which allows to trim the offsets to avoid including the leading whitespace. (cf https://github.com/huggingface/transformers/blob/11505fa139f6eb4896d7f1d181da1e026878d489/src/transformers/convert_slow_tokenizer.py#L293)

For more info about the bug @LysandreJik mentioned, you can have a look at https://github.com/huggingface/transformers/issues/9637. The fact that the leading whitespace is not included when you load your tokenizer from the hub is indeed a bug, and it leads to wrong offset mappings in many situations.<|||||>Thanks for the context! Closing as a duplicate of #9637