word_to_tokens method of XLNetTokenizerFast not behaving correctly
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3


### Who can help
@LysandreJik 


## Information
The `word_to_tokens` method of `XLNetTokenizerFast` seems not behaving correctly.

## To reproduce
Code below for example
```py
batch_claim = [
    ['Colin', 'Kaepernick', 'became', 'a'],
    ['Tilda', 'Swinton', 'is', 'a', 'vegan', '.']
]
batch_evidence = [
    ['He', 'remained', 'the', 'team', "'s", 'starting', 'quarterback'],
    ['Katherine', 'Matilda', '`', '`', 'Tilda', "''", 'Swinton', '-LRB-', 'born', '5', 'November', '1960']
]

tokenizer = XLNetTokenizerFast.from_pretrained('xlnet-base-cased')
# tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', add_prefix_space=True)
tokenized = tokenizer(
    batch_claim, batch_evidence,
    padding=True, truncation='do_not_truncate', is_split_into_words=True, return_tensors='pt'
)
print(tokenized)
print(tokenized.word_to_tokens(0, 0, 0))
```

gives None. (Maybe it's because that `XLNetTokenizer` pads on the front that causes this misbehavior?)

Output:
```
{'input_ids': tensor([[    5,     5,     5,     5,     5,     5,     5,     5,     5,     5,
             5,     5,     5,     5,  8041,  2066,    93,  1371,  9797,   403,
            24,     4,    69,  1493,    18,   230,    17,    26,    23,  1541,
          6217,     4,     3],
        [15731,  1011, 22588,   577,    27,    24, 28629,    17,     9,     4,
         17067,  6883,   902,  1011,  2651,  2651, 15731,  1011,    17,    12,
         22588,   577,    17,    13,  1039, 12573,    13,  1094,   306,   704,
          2726,     4,     3]]), 'token_type_ids': tensor([[3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 2],
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 2]]), 'attention_mask': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 1],
        [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 1]])}
None
```

<!-- A clear and concise description of what you would expect to happen. -->

Indeed, thanks for reporting. This is related to https://github.com/huggingface/tokenizers/issues/552<|||||>Thanks! Could you please indicate the time this could be fixed? I'll decide whether to align it locally haha..<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.