@BramVanroy - thank you!

If your model is specifically finetuned on single words, then it should be fine

So, when you say the above, I am assuming something like this:
    #new_vocab contains words specific to my dataset
    new_vocab = open('/tmp/new-vocab.txt', 'r').read().split('\n')
    new_vocab.remove('')

    print('Adding new tokens to vocab')
    tokenizer.add_tokens(new_vocab)
    model.resize_token_embeddings(len(tokenizer))

    training_args = TrainingArguments(
    "test-mlm",
    evaluation_strategy = "epoch",
    learning_rate=2e-5,
    weight_decay=0.01,
    num_train_epochs=20.0
)
   trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=lm_datasets["train"],
    eval_dataset=lm_datasets["validation"],
    data_collator=data_collator,
)  

Basically, adding the new words to the vocab and fine-tuning RoBerta further using MLM only.