Why is the NSP task in BERT inconsistent or ineffective? The NSP task is one of the two tasks in BERT which has been revolutionizing NLP, but many pretrained models abandoned that task, for instance
First, XLNet removed NSP

XLNet-Large does not use the objective of next sentence
prediction [10] as it does not show consistent improvement in our ablation study

In RoBERTa, researchers found that

removing the NSP loss matches or slightly improves downstream task performance

It states in ALBERT that

SOP primary focuses on inter-sentence coherence and is designed to address the ineffectiveness (Yang et al., 2019; Liu et al., 2019) of the next sentence prediction (NSP) loss proposed in the original BERT

They all just showed that the NSP is either inconsistent or ineffective, but don't explain why. I wonder why the task is ineffective?

I find an explanation here:

ALBERT conjectures that NSP was ineffective because it’s not a difficult task when compared to masked language modeling. In a single task, it mixes both topic prediction and coherence prediction. The topic prediction part is easy to learn because it overlaps with the masked language model loss. Thus, NSP will give higher scores even when it hasn’t learned coherence prediction.

And the NSP added some noise to the masked language model[1].
