So far, I’ve been using pre-trained models. For my task, it seems that I am required to perform pre-training on GLUE task just to see how it performs. I wanted to confirm what modifications need to be done to do this ? I’m not sure about using the same tokenizer.
I want to randomly initialize it and train on GLUE task. Additionally, if you some tips on effectively doing it when not using fine-tuning weights, please share ?