I am using a RoBERTa based model for pre-training and fine-tuning.
To pre-train, I use RobertaForMaskedLM with a customized tokenizer . This means I used my tokenizer in the LineByLineTextDataset() and pre-trained my model for masked language modeling.
However, for fine tuning, When I want to use my dataset with labels for a classification task, I think I must use my customized tokenizer before feeding my data to the model for fine tuning.
My question is, How can I use my tokenizer to prepare the data and fine tune my pre-trained model?