run_language_modeling.py does not output vocab/config/etc files until training completes
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
GPT2

Language I am using the model on (English, Chinese ...):
English

The problem arises when using:
* the official example scripts: 
I began training using the following command (from Jupyter Lab) - output_100k is the specified (new) output folder for my fine tuned model (which in being trained on 100k US patents):
`!python run_language_modeling.py \
    --output_dir=output_100k \
    --model_type=gpt2 \
    --model_name_or_path=gpt2 \
    --block_size 100 \
    --per_device_train_batch_size 3 \
    --do_train \
    --train_data_file=./train_100k.txt \
    --do_eval \
    --eval_data_file=./test_100k.txt`

This works fine; HOWEVER, no instructions are given in the example [README](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) on how to resume training later on. 
**Including this information would be extremely helpful**. After some personal research, I deduced (perhaps incorrectly?) that I should point 'model_name_or_path' at the output_100k checkpoint folder like so (note: the README also doesn't specify whether or not it's okay to keep the original output-dir name, so I made a new one): 
`!python run_language_modeling.py \
    --output_dir=output_100k_resumed \
    --model_name_or_path=./output_100k/checkpoint-12500 \
    --block_size 100 \
    --per_device_train_batch_size 3 \
    --do_train \
    --train_data_file=./train_100k.txt \
    --do_eval \
    --eval_data_file=./test_100k.txt`

This raises the following error:
`OSError: Model name './output_100k/checkpoint-12500' was not found in tokenizers model name list (gpt2, gpt2-medium, gpt2-large, gpt2-xl, distilgpt2). We assumed './output_100k/checkpoint-12500' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.`

I checked the directory, and indeed, there are no vocab.json, merges.txt, etc files included in the specified output directory from the initial training.

It appears that **run_language_modeling.py** does not output these files until the **end** of training (an examination of other models training using the script shows that these files are present in completed training sessions.

**What am I doing wrong?** Or is this a bug in the script itself? I'd imagine these vocab/merges files can be outputted relatively early in training. Note, when training on a new dataset starts, a file with the name of the form `cached_lm_GPT2Tokenizer_[SOME NUMBER]_train_clean.txt.lock` (and one without the .lock) is generated. Perhaps this contains some information that I'd need to resume training? Regardless, I see no documentation explaining this file's purpose linked in the README, which - again - could definitely provide more context.


The tasks I am working on is:
* my own task or dataset: (give details below)
The idea is to build an effective context-aware text generator for legal purposes. The dataset simply consists of a bunch of patent document texts.

## To reproduce

Steps to reproduce the behavior:
**See description above**

## Expected behavior

Model resumes training at the specified checkpoint.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-5.3.0-59-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No modifications to script, but yes, this computer uses GPU.
- Using distributed or parallel set-up in script?: No.


**NOTE:** Currently retraining. At the 500-it checkpoint save point, the following is outputted:
`"loss": 3.162974836349487, "learning_rate": 4.207104345068189e-05, "epoch": 0.47573739295908657, "step": 500}
06/23/2020 10:25:35 - INFO - transformers.trainer -   Saving model checkpoint to ./output_100k_run2/checkpoint-500
06/23/2020 10:25:35 - INFO - transformers.configuration_utils -   Configuration saved in ./output_100k_run2/checkpoint-500/config.json
06/23/2020 10:25:35 - INFO - transformers.modeling_utils -   Model weights saved in ./output_100k_run2/checkpoint-500/pytorch_model.bin
/home/b/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
`

I'm particularly concerned about that last part:
`/home/b/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.`

Is that not implemented by default? Do I have to pass in a specific argument to fix it? Or is this something that should be ignored? 

Not sure if this is related to the above bug, but... maybe?

**NOTE 2:** The contents of the output folder at this point (500 iterations of training):

`checkpoint-500`

And the contents of a similar model output dir that I trained fully:
`checkpoint-1000  config.json        special_tokens_map.json  vocab.json
checkpoint-1500  merges.txt         tokenizer_config.json
checkpoint-500   pytorch_model.bin  training_args.bin
`
Hopefully that highlights the issue - all those supplementary files don't seem to be being saved until the end of training.
<|||||>Hi, I'm also having this issue where `run_language_modeling.py` is not creating the files needed to resume the training until it finishes and by then you wont need the files to resume it as it will have finished, also cant generate text until the vocab,tokens,etc are created as those files are needed for the model to work correctly, in my case im using the code from [this repository](https://github.com/itsuncheng/fine-tuning-GPT2) but technically it's the same code found on the [example/language-modeling](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) folder with just some extra information on the readme file.
Have you guys found any way to make it work ?<|||||>Looks like in #3921 a similar issue was fixed in https://github.com/huggingface/transformers/commit/c81152600452ad1bec4ab705356788d29a3573ee by adding `tokenizer.save_pretrained(training_args.output_dir)` to the end of the script. Could this be done for every checkpoint, instead of just at the final output step? If not, how are the checkpoints supposed to be used?

Thanks!<|||||>> I'm particularly concerned about that last part:
> `/home/b/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.`

I have the same warning. Sounds bad for me...
Can I just ignore it? 

```bash
UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
```
<|||||>This appears to still be an issue. Has anyone found a solution?<|||||>@apteryxlabs Here's a not-so-eloquent short term fix. I'm just forcing the optimizer to load around line 515 of trainer.py. optimizer = torch.load('path/to/optimizer.pt'). Again, not eloquent but it seems to work. Still trying to figure out why the optimizer isn't loading properly earlier on. 
<|||||>> @apteryxlabs Here's a not-so-eloquent short term fix. I'm just forcing the optimizer to load around line 515 of trainer.py. optimizer = torch.load('path/to/optimizer.pt'). Again, not eloquent but it seems to work. Still trying to figure out why the optimizer isn't loading properly earlier on.

Hi, thanks for the hint. Would you please elaborate which line around line 515? I saw an optimiser.pt appearing at line 623, another 463, not sure which one you are referring to. Also, is it just setting the path of this optimizer.pt brutally to the file under the /checkpoint-x directory?<|||||>I am having the same issues: not saving supplementary files at checkpoints and also the warning about loading the optimizer. Has anyone found a solution?<|||||>It seems that supplementary file saving at checkpoints has been fixed (I am now seeing checkpoint saving in finetune).

But I am still seeing 
"Warning: Please also save or load the state of the optimzer when saving or loading the scheduler.  warnings.warn(SAVE_STATE_WARNING, UserWarning)"


Steps to reproduce:
1) clone transformers into new director
2) cd transformers && pip install .e; cd examples && pip install -r requirements.txt
3) cd seq2seq && ./finetune_t5_bart_tiny.sh

Observe that the warning is printed:

../python3.8/site-packages/pytorch_lightning/utilities/distributed.py:37: UserWarning: Could not log computational graph since the `model.example_input_array` attribute is not set or `input_array` was not given 
warnings.warn(*args, **kwargs)
.../python3.8/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
  warnings.warn(SAVE_STATE_WARNING, UserWarning)

(There is both the optimizer warning and the computational graph logging warning)<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
