Meaning of run_lm_finetuning.py output
## ❓ Questions & Help
Is there documentation somewhere about what the various output files that get created when running `run_lm_finetuning.py` are and what the meaning of their contents is? Concretely, what are the files and directories:
```
added_tokens.json  
checkpoint-50/
checkpoint-100/    
checkpoint-150/  
checkpoint-200/  
checkpoint-250/
checkpoint-300/  
checkpoint-350/  
checkpoint-400/  
config.json    
eval_results.txt  
merges.txt        
pytorch_model.bin     
runs/   
special_tokens_map.json  
tokenizer_config.json  
training_args.bin
vocab.json
```
and what is the meaning of their contents? the `checkpoint` directories contain:
```
config.json  
pytorch_model.bin  
training_args.bin
```

and `runs/Dec03_09-15-51_MACHINENAME` contains:
```
events.out.tfevents.20414.0
```

## Finetuning Setup
* Model: roberta-base
* Language: english
* OS: Ubuntu 18.04.3
* Python version: 3.7.3
* PyTorch version: 1.3.1+cpu
* PyTorch Transformers version (or branch): 2.2.1, I believe
* Using GPU ? No
* Distributed of parallel setup ? No
* Script inputs:
```
python run_lm_finetuning.py \
--output_dir=$OUTPUT_DIR \
--model_type=roberta \
--model_name_or_path=roberta_base \
--do_train \
--train_data_file=$TRAIN_FILE \
--do_eval \
--eval_data_file=$TEST_FILE \
--mlm \
--no_cuda
```

Thanks in advance!

I have a similar question. When using default settings, does anything change in the tokenizer? Is the tokenizer fine-tuned in anyway (or is any vocabulary added)? In other words, is the vocab.txt of use in any way, when using the default tokenizer? If not, I assume that you only need the `pytorch_model.bin` file and you're good to go?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
