What is the relationship between `run_lm_finetuning.py` and the scripts in `lm_finetuning`?
## ❓ Questions & Help

It looks like there are now two scripts for running LM fine-tuning. While `run_lm_finetuning` seems to be newer, the documentation in `lm_finetuning` seems to indicate that there is more subtlety to generating the right data for performing LM fine-tuning in the BERT format. Does the new script take this into account?

Sorry if I'm missing something obvious!
Hi! The folder `lm_finetuning` is especially targeted at BERT. It gives details on two different losses that were used to pre-train BERT: the masked language modeling objective (MLM) and the next sentence prediction objective (NSP). It gives several insights to BERT's fine-tuning.

The file `run_lm_finetuning`, on the other hand, showcases how to fine-tune language modeling on several models: BERT, GPT, GPT-2, and RoBERTa. It only uses a single objective; MLM for BERT and RoBERTa and CLM (causal language modeling) for GPT and GPT-2.<|||||>How can we fine-tune on the next sentence prediction task? I did not find the `lm_finetuning` files. Thank you.<|||||>Hi @JiajunBao, these scripts were community maintained and have since been removed. We do not have any script working on the next sentence prediction task. I believe the `lm_finetuning` files were last up to date in 1.1.0, so you may look [here](https://github.com/huggingface/transformers/tree/1.1.0/examples/lm_finetuning).<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
