Hi Bram. Thank you very much for your kind reply. After seeing your reply I rethink of the so called further pre-training that I want to do. I realize that I can actually just directly train, for example, BertForSequenceClassification model and by fitting the model to my data I am actually already doing the further pre-training and classification at the same time, because all the model parameters will be updated according to my new data(If I don’t intentionally freeze the weight update for all the transformer layer). Am I understanding this correctly ?