`offset_mapping` is strange for non-ascii token
### System Info

```shell
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
```


### Who can help?

@SaulLu

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. Use BertTokenizerFast
2. tokenize the text 
text = "The Dungan Revolt ( 1862–77 ) or Tongzhi Hui Revolt ( , Xiao'erjing : توْحُوِ بِيًا/لُوًا ,  ) or Hui ( Muslim ) Minorities War was a mainly ethnic and religious war fought in 19th-century western China , mostly during the reign of the Tongzhi Emperor ( r. 1861–75 ) of the Qing dynasty ."
outputs = self.tokenizer(text,
                                    padding="max_length",
                                    truncation=False,
                                    max_length=self.config.max_seq_length,
                                    return_offsets_mapping=True)
3. The `offset_mapping` missed some subtokens, for example, the subtoken at the char position 79. 

<img width="83" alt="截屏2022-04-20 下午8 46 55" src="https://user-images.githubusercontent.com/39556019/164233684-192d1c32-0f67-4198-8e59-e60426fce32d.png">


### Expected behavior

```shell
the offset_mapping should be:
[... 
 (70, 71),
 (71, 72),
 (73, 74),
 (76, 77),
 (78, 80), # this line ****
 (81, 82),
 (83, 84),
... ]
```

The environment info is:
- `transformers` version: 4.18.0
- Platform: Linux-4.15.0-136-generic-x86_64-with-glibc2.27
- Python version: 3.9.12
- Huggingface_hub version: 0.5.1
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>