@Yuti In my code, I have this line which allows me to adjust the masked language probability:
from transformers import DataCollatorForLanguageModeling
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm_probability=0.15)

However, if I adjusted it, I am unsure how to do it for the emojis.