Some weights of AlbertForPreTraining were not initialized from the model checkpoint at albert-base-v2 and are newly initialized: ['sop_classifier.classifier.weight', 'sop_classifier.classifier.bias']
Albert leverage sentence coherence predict loss(SOP) to optimizer its parameters, and I wanna employ it to score the coherence between two sentences. But when I use AlbertForPreTraining to load albert-xxlarge-v2 checkpoint, it reminds me that:
_Some weights of AlbertForPreTraining were not initialized from the model checkpoint at albert-base-v2 and are newly initialized: ['sop_classifier.classifier.weight', 'sop_classifier.classifier.bias'] 
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference._ 

I try many times and find the output of each time is different, this means that the final linear layer for classification has not been loaded and just initial by random. I wanna know how to leverage pretrain SOP classification without fine-tuning?

Hope to get response rapidly.
I'm not sure we have a completely fine-tuned SOP classification model. My best advice is to try out different models and see which model not randomly allocates the weights for those layers.