Wrong offsets_mapping in T5TokenizerFast 
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.1
- Platform: Linux-4.9.0-14-amd64-x86_64-with-debian-9.13
- Python version: 3.6.10
- PyTorch version (GPU?):  1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help @patrickvonplaten, @mfuntowicz 
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 ray/raytune: @richardliaw @amogkam
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using: T5

## To reproduce
See comments in the code snippet.
```python
from transformers import T5TokenizerFast


def test_offset_mapping():
    """This test fails and therefore we know that there is a bug in offset_mapping mechanism.
        We try to tokenize the sentence 'This is a test sentence' and notice to issues:

        1. The tokenizer tokenizes it to ['This', 'is', '', 'a', 'test', 'sentence']
            which means that it has redundant empty string in position 2.
        2. The offset mapping maps to ['This', 'is', 'a', 'a', 'test', 'sentence']
            replacing the empty string with redundant 'a'.

    """
    tokenizer = T5TokenizerFast.from_pretrained('google/t5-v1_1-base')

    s = "This is a test sentence"
    tokenized = tokenizer(s, return_offsets_mapping=True)
    
    decoded_tokens, tokens_from_offset_mapping = [], []
    for token_index, offset_mapping in enumerate(tokenized['offset_mapping']):
        decoded_token = tokenizer.decode(tokenized['input_ids'][token_index])
        if decoded_token != tokenizer.eos_token:
            decoded_tokens.append(decoded_token)
            tokens_from_offset_mapping.append(s[offset_mapping[0]:offset_mapping[1]])

    error_msg = f"Wrong offset mapping for '{s}'! \n" \
                f"Maps to:          {tokens_from_offset_mapping}\n" \
                f"Instead of:       {decoded_tokens}"
    assert decoded_tokens == tokens_from_offset_mapping, error_msg


if __name__ == "__main__":
    test_offset_mapping()
```

## Expected behavior
```
AssertionError: Wrong offset mapping for 'This is a test sentence'! 
Maps to:          ['This', 'is', 'a', 'a', 'test', 'sentence']
Instead of:       ['This', 'is', '', 'a', 'test', 'sentence']
```

@patrickvonplaten @n1t0 do you have any advice on this? The T5 tokenizer tokenizes the sentence as follows:
```
['▁This', '▁is', '▁', 'a', '▁test', '▁sentence']
```

Unfortunately the offset mapping point to both  '▁' and 'a' being at `(8, 9)`, as the following suggests:
```
'offset_mapping': [(0, 4), (5, 7), (8, 9), (8, 9), (10, 14), (15, 23), (0, 0)]
                                    ^---- & ^---- here 
```

How should one map this encoding back to the initial sequence?<|||||>@patrickvonplaten @n1t0 - did you have a chance to look at this?
Thanks!<|||||>Hi @zorikg! Thank you for reporting this issue. This is related to https://github.com/huggingface/transformers/issues/9637 concerning the offset mappings bug.

The fix for this bug is tricky to deploy, but we are working on it, and I expect it to be available in the coming weeks.<|||||>Thanks @n1t0, I wondered if there have been any progress on this? Any expectation for when the fix will be avail? Thanks!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>@zorikg Using the last few versions of `transformers`, you can instantiate your tokenizer as follow:
```python
tokenizer = T5TokenizerFast.from_pretrained('google/t5-v1_1-base', from_slow=True)
```

This will force the conversion from the slow tokenizer, thus using the fixed version.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>I am getting some difference between these 2 tokenizers is this solved?