@neuralpat thank you for your answer, but I am afraid that my post was not clear.
I have an annotated dataset to fine tune BERT for Sequence Labeling (Name Entity Recognition), and a much larger dataset that is not annotated but it is from the same domain. I want to continue the pre-training of BERT on the dataset of the associated domain, to see whether it will help BERT perform better in NER.
Maybe, I have to rewrite my OP.