#1 is very computationally expensive and increasingly unnecessary.

Did you mean training from scratch here? I was referring to using the pretrained model without any fientuning, so just download and apply distillbert for instance. So that would be the minimum amount of work?

So I think maybe #2 and #3 are the same thing.

The big difference here is the amount of labelled vs unlabelled data (1M vs 50K). If I limit myself to tuning on labelled data, the masking process will have a much smaller corpus to learn from. This is why I would want to do it first with the 1M unlabelled records, which will create a stronger general vocabulary for the model. Then I would use the final 50K labelled records to finetune further specific to the labels. In a nutshell, if I skip #2 the model will have a smaller vocabulary than if I include it.