Checkpoints not saved
There is an option `save_checkpoints_steps` that seems to control checkpointing. However, there is no actual saving operation in the `run_*` scripts. So, should we add that functionality or remove this argument?
In the `run_squad.py`script, I added the following lines after the training loop:

```
logger.info(***** Saving fine-tuned model *****)
output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
if n_gpu > 1:
    torch.save(model.module.bert.state_dict(), output_model_file)
else:
    torch.save(model.bert.state_dict(), output_model_file)
```

The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.

I do not use the other `run_*` scripts but this may be applicable as well.

Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.<|||||>You are right this argument was not used. I removed it, thanks. These examples are provided as starting point to write training scripts for the package module. I don't plan to update them any further (except fixing bugs).<|||||>> In the `run_squad.py`script, I added the following lines after the training loop:
> 
> ```
> logger.info(***** Saving fine-tuned model *****)
> output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
> if n_gpu > 1:
>     torch.save(model.module.bert.state_dict(), output_model_file)
> else:
>     torch.save(model.bert.state_dict(), output_model_file)
> ```
> The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.
> 
> I do not use the other `run_*` scripts but this may be applicable as well.
> 
> Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.

what is your result on adversarial-squad?<|||||>At that time I got:
**AddSent**
BERT base 58.7 EM / 66.2 F1
BERT large 65.5 EM / 71.9 F1

**AddOneSent**
BERT base 67.0 EM / 74.7 F1
BERT large 72.7 EM / 79.1 F1



<|||||>> At that time I got:
> **AddSent**
> BERT base 58.7 EM / 66.2 F1
> BERT large 65.5 EM / 71.9 F1
> 
> **AddOneSent**
> BERT base 67.0 EM / 74.7 F1
> BERT large 72.7 EM / 79.1 F1

Thanks a lot! Do you release your paper? i want to cite your result and paper in my paper.<|||||>Unfortunately it was not part of a paper, just preliminary results.