Dose the file /examples/run_lm_finetuning.py provide a demo to pre-train a BERT
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hi, I'm curious that if the file run_lm_finetuning.py can pre-train a BERT. But if so, why the file named finetuning instead of pre-training? Thanks!
Hi, BERT uses the NSP task as well as the MLM task during its pre-training phase. The `run_lm_finetuning.py` script only does MLM so you would need to modify it to pre-train BERT the same way it was done in the paper.