Machine Learning, NLP, Bert, Albert, Transformers.
for the fact that both sentences go in inverse order. Composition of positive and negative training pairs in BERT and in ALBERT It was shown that models trained with the NSP objective cannot accurately solve SOP tasks while models trained with the SOP objective perform well on NSP problems. These