Tokenizstion
Hi all, help me please. I'm trying to solve multilingual WiC problem using XLM-R. 
I have a word and a sentence, that has this word, but may be in different form. So I want to find the position of its token ids in an encoded sentence. 
There problems with Chinese, cause they don't have space between many symbols and tokenizer can encode a pair of symbols differently.
So in short:
- in public open-source projects like this you can say "Hi all" or "Hi folks", this way you'll address contributors of all genders
- also, do you mind closing this issue and opening a thread on the forum at https://discuss.huggingface.co? We keep the issues for bug reports and feature requests which this issue is not.
- last note for your future issues on open-source projects: when there is an issue template like here you should fill it that's typically the first thing the maintainers will ask you if you didn't do it.

This looks like a nice project good luck with it and I hope you'll succeed!