Hi,
you should try to fine-tune the model first. I could only image a few scenarios where it makes sense to train a model from scratch: vocab should be very different, e.g. when your domain are historical texts (or digitized texts with ocr errors…).
And you should have a look at the SciBERT paper (https://arxiv.org/abs/1903.10676 6) - for some datasets the difference between “normal” BERT and SciBERT is very close…
What are your downstream tasks for evaluation btw.