BERT pre-training using only domain specific text
BERT is pre-trained using Wikipedia and other sources of normal text, but my problem domain has a very specific vocabulary & grammar. Is there an easy way to train BERT completely from domain specific data (preferably using Keras)?
 
The amount of pre-training data is not issue and we are not looking for the SOTA results. We would do fine with a smaller scale model, but it has to be trained from our data.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
