Hi aberquand,
I don’t think you can use the pre-trained weights with your domain-specific vocabulary.
[I am not an expert, and I’ve only used BERT not RoBERTA, and I didn’t use the Trainer, so I could be wrong.]
If I understand it correctly, the way the weights learn is dependent on the particular vocabulary.
I suggest you use the pre-trained vocabulary as well as the pre-trained weights.
How different is your vocabulary from the original RoBERTA vocabulary?   I would expect 700000 sentences would be enough to do fine-tuning, but probably not enough to train from scratch.
Did you know, if you use Google Colaboratory you can get a limited amount of GPU-time for free.  It generally maxes out after about 7 hours each day.  Colab uses Jupyter notebooks.
For more information on tokenizer vocabulary, I recommend Chris McCormick’s blogs, eg https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/ 5