Agreed, BioBERT uses in fact the BERT-base vocabulary.
Also interested by a better classification of what “pre-training” encompasses, perhaps could use "post-training’ or ‘further training’ as suggested by @BramVanroy.
Another relevant work for anyone interested to compare the effect of using BERT-base vocab or your own vocab: SCIBERT (https://arxiv.org/pdf/1903.10676.pdf 11) further trained a BERT Base model with Scientific data, either using BERT’s vocab or their own SciVocab.
But based on your previous comments and the size of my training set, I’ll start by building on top of roberta-base. Thanks a lot for your help @rgwatwormhill and @BramVanroy, finally moving forward!