Definitely finetune IMO. Will be much faster and better results. Would you rather teach a third grader how to predict the next word on your dataset or a newborn?
One argument for from scratch would be  more control. It is less likely your from scratch model says something racist or otherwise wrong if it is trained on just your (presumably friendly) data.