How to improve language model ex: BERT on unseen text in training? so I am using pre-trained language model for binary classification. I fine-tune the model by training on data my downstream task. The results are good almost 98% F-measure.
However, when I remove a specific similar sentence from the training data and add it to my test data, the classifier fails to predict the class of that sentence. For example, sentiment analysis task

"I love the movie more specifically the acting was great"

I removed from training all sentences containing the words " more specifically" and surprisingly in the test set they were all misclassified, so the precision decreased by a huge amount.
Any ideas on how can I further fine-tune/improve my model to work better on unseen text in training to avoid the problem I described above? (of course without feeding the model on sentences containing the words "more specifically")
Note: I observed the same performance regardless of the language model in use (BERT, RoBERTa etc).

So now you have one model that works OK on sentences that contain "more specifically", and one that works on other sentences.
Then the right predictor would seem to be:
model1(sentence) if 'more specifically' in sentence else model2(sentence)

If your goal is simply to make good predictions, then that would seem to ge good enough. Maybe the condition can be generalised a little to regexp 'more \w+ly ', or maybe not. I suspect that the non-grammatical nature of the sentence may be part of the issue.
Alternatively, if you want to know you have made a single neural net model, you could take the second model, then retrain it with the movies test set and a low learning rate. That might improve it, without making it forget its success on these sentence. Or not!
