I added a config.json file manually in the same folder as the tokeniser files, i.e.
{
“model_type”: “roberta”,
“vocab_size”: 52000,
“type_vocab_size”: 1
}
I’m not sure there’s a step-by-step guide but in general if you AutoTokenizer.from_pretrained() rather than RobertaTokenizerFast.from_pretrained() then you need a config.json containing at least the model_type so Auto knowns which model family you’re using. Plus any other parameters that differ to the roberta defaults (such as the vocab size).
You need to do the same thing for your model, i.e. in the mlm script it loads AutoConfig.from_pretrained(config_name) to initialise the actual model parameters used in training (number of heads etc). You need to create your own config.json containing the parameters from RobertaConfig so AutoConfig can load them (best thing to do is start by copying the config.json for Roberta from the model hub then modify as required).