Incorrect Tokenization behavior when working with Hindi using RobertaTokenizer
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: https://github.com/huggingface/transformers@534f6eb9f1f8910a4912ccccd79f1f974c625168
- Platform: Linux
- Python version:  3.8.10
- PyTorch version (GPU?): 1.9.0 (Not using GPU)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


## Information

### Problem
[Our model](https://huggingface.co/flax-community/roberta-hindi) is tokenizing text in way longer sequences, more like a character level lm, than as sub-word level lm.

### Diagnosis
Tokenizer is at fault, when it tokenizes on a word with matra, it always breaks matra, i.e. diacritics into a seperate token, as in `'संसद' => [' स', 'ं', 'सद']`, where as a more plausible tokenization should be `'संसद' => ['सं', 'सद']`
For more details one can look at the analysis csv [here](https://gist.github.com/amankhandelia/f8192a6782714fab6537dc29aef8fbfc) where I have done the analysis for five different models. Given the pattern observed in all RoBERTa models I believe that the [BBPE tokenizer](https://arxiv.org/pdf/1909.03341.pdf) used in case of [RoBERTa-hindi-guj-san](https://huggingface.co/surajp/RoBERTa-hindi-guj-san), [HindiBERTa](https://huggingface.co/mrm8488/HindiBERTa) and [our model](https://huggingface.co/flax-community/roberta-hindi) is not upto the task, or need some modifications to work on Hindi. 

The tasks I was working on was Roberta Hindi LM as part of the recent flax community sprint. Although the sprint has ended, I was analyzing the tokenizers in order to learn how these model works, when I stumbled upon this observation, so this bug report is more of question than an explicit bug.
Is this the expected behavior of BBPE, if so, what is a possible remedy, can I use different tokenizer with Roberta instead using the RobertaTokenizer?

cc: @LysandreJik, @patil-suraj

Keeping Suraj in cc, as he might be able to better communicate the problem I am facing, with the HF team.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.