msamogh:

they should have obviously used the Next Sentence Prediction task


Why? Follow-up papers have shown that NSP does not contribute much, if anything at all. (RoBERTa dropped it completely, Albert uses sentence order prediction.) The authors were likely aware of these findings and did not feel the need to include this task. Note, by the way, that the BERT weights also do not include the NSP weights.
To make this work, you’d have to finetune on this task specifically. You can use the pretrained model as a feature extractor (frozen) and add a classification head on top, or finetune end-to-end, or unfreeze gradually. If you have plenty of data and compute, you can train from-scratch but as you note that may not be feasible.