Wrong RoBERTa configuration
https://github.com/huggingface/transformers/blob/c2393cad085e3875ee2206d917d46d15e50602a3/src/transformers/models/roberta/configuration_roberta.py#L108

Must match the tokenizer vocab. size, i.e., `50265`.

Others also mentioned this: https://discuss.pytorch.org/t/hugging-faces-roberta-config-and-tokenizer-do-not-have-matching-vocabulary/134868
cc @ArthurZucker <|||||>Thanks for pointing this out and opening a PR! <|||||>Hey @ArthurZucker @amyeroberts , as i still see that  #23389  is not merged , so can I fix this and create a new PR? <|||||>Sure! Would be great if you can checkout his branch to include the work he has done 😉 <|||||>fix #23863 