I specifically wanted to do the additional pretraining.
It is one if the steps used in some papers in order to adjust the distribution.
The masked lm and next sentence classifier are available in huggingface, but I haven’t seen any training examples.