I would like to fine-tune RoBERTa on a domain-specific English-based vocabulary.
For that, I have done a TF-IDF on a corpus of mine, and extracted 500 words that are not yet in RoBERTa tokenizer.
As they represent only 1 percent of total tokenizer length, I don’t want to train the tokenizer from scratch.
So I just did :
tokenizer.add(['word1', 'word2'])
model.resize_token_embeddings(len(tokenizer))

BUT I see 2 problems related to BPE :

My words are not truncated into sub-words. Is it fine enough ?
There are no " Ġ " (\u0120) in my list. Should I add them manually ?*

I am adding that I could not find any precise answer to this question (that many of us have) : see Usage of Ġ in BPE tokenizer · Issue #4786 · huggingface/transformers · GitHub