Hi @adamh
You can use the run_language_modeling script here 16 to finetune the pre-trained model for ex BertForMaksedLM.
Then you should be able to load the model using BertForSequenceClassification model which will take the base model and add a classification head on  top, which you can then fine-tune for classification.