Yes, your pretraining approach with DistilBERT is perfectly valid since in that application you’re simply using the model architecture without any knowledge distillation from a teacher. (i.e. “under the hood” it’s no different to pretraining BERT or GPT-2 etc from scratch).

So my training would still be valid but I should expect less accuracy than if I did the pretraining through distillation. Is this correct?

In my experience it is generally true that distilled models perform better than training the same model from scratch. Out of curiosity, what is your use-case? You might find that someone has already pretrained a language model for your domain / language which would be a better starting point than training from scratch