Hi @ayalaall, did further training on NSP task on a custom corpus improve performance of the embeddings?
My use case: Further train bert-base-uncased model on custom text corpus, then use it to get sentence embeddings
I tried just using the MLM task for training but the embedding performance was even poorer than the bert-base model, is the NSP task very important when the use case is embedding generation?