New run_language_modeling.py continuing trainng
I tried using new script for Language Modeling to train model from scratch.

I was training model with old script. When I tried continuing using new one, there is no option for `--should_continue`.

Does `--overwrite_output_dir` makes it train from scratch if i use same directory for `--model_name_or_path` and `--output_dir`?

So, I am continuing training with old script.
Yes, we removed the `--should_continue` flag. You can just use your latest checkpoint as `--model_name_or_path`

Let us know if this helps<|||||>Yup, The new script is really clean and understandable. To continue training, I should use `--overwrite_output_dir`?  Because, when I point to last checkpoint directory using `-- model_name_or_path`, It asks me for `--output_dir` which must be empty,  so I had to switch between two directories. <|||||>Yes you would use `--overwrite_output_dir` (it was implicitly added before)

You would do:
```
--model_name_or_path ./model_name/checkpoint-9000
--output_dir ./model_name
--overwrite_output_dir
```

If this is a frequent request we could add it back but I feel like it was more confusing than anything.<|||||>Yes would like to get this feature back. <|||||>I am using [TrainingArguments](https://huggingface.co/transformers/main_classes/trainer.html#trainingarguments) but couldn't continue training from last checkpoint. It is starting from step 0.  Any suggestion?

```
training_args = TrainingArguments(
    output_dir=last_checkpoint,
    overwrite_output_dir=True,
...
)
 trainer = Trainer( model=model, #roberta config model
           args=training_args,
...
)
```

e.g., `last_checkpoint="/saved_model/checkpoint-20000/"`
@julien-c 