Added additional training utils for run_squad.py
Added different LR schedules and hyperparameters for beta1 and beta2 for AdamW
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=h1) Report
> Merging [#1766](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1c542df7e554a2014051dd09becf60f157fed524?src=pr&el=desc) will **decrease** coverage by `1.38%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1766/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1766      +/-   ##
==========================================
- Coverage   84.03%   82.65%   -1.39%     
==========================================
  Files          94       94              
  Lines       14032    14032              
==========================================
- Hits        11792    11598     -194     
- Misses       2240     2434     +194
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3RmX3B5dG9yY2hfdXRpbHMucHk=) | `9.85% <0%> (-83.1%)` | :arrow_down: |
| [transformers/tests/modeling\_tf\_common\_test.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2NvbW1vbl90ZXN0LnB5) | `79.78% <0%> (-17.03%)` | :arrow_down: |
| [transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3JvYmVydGEucHk=) | `59.41% <0%> (-12.36%)` | :arrow_down: |
| [transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3hsbmV0LnB5) | `71.18% <0%> (-2.44%)` | :arrow_down: |
| [transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2N0cmwucHk=) | `94.24% <0%> (-2.22%)` | :arrow_down: |
| [transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/1766/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX29wZW5haS5weQ==) | `80.66% <0%> (-1.34%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=footer). Last update [1c542df...5f1eca9](https://codecov.io/gh/huggingface/transformers/pull/1766?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Ok with that. Merging.
cc @LysandreJik and his refactoring of the `run_squad` example.<|||||>Actually seems like I can't push a merge commit on your branch to fix conflict.
Will close the PR later unless you can fix it.