Machine Learning, NLP, Bert, Albert, Transformers.
Prediction BERT focuses on mastering two objectives when pretraining: masked language modeling (MSM) and next sentence prediction (NSP). In general, MSM was designed to improve BERT’s ability to gain linguistic knowledge and the goal of NSP was to improve BERT’s performance on particular downstream