Adding --do_lower_case option
In SQuAD 1.1 using the uncased BERT model, **--do_lower_case** option must be used for initialization of a tokenizer.
Omitting this optional argument, the performance in development set would be lower around 8-9pt in both EM and F1 (In the official setting described in readme, the actual metrics were 72.09 (EM)/81.84 (F1) we confirmed).
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=h1) Report
> Merging [#4000](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4e817ff41885063e08bb3bcd63e5adfd835b9911&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4000/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4000      +/-   ##
==========================================
- Coverage   78.44%   78.44%   -0.01%     
==========================================
  Files         111      111              
  Lines       18518    18518              
==========================================
- Hits        14527    14526       -1     
- Misses       3991     3992       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4000/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68.49% <0.00%> (-0.37%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4000/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.59% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4000/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=footer). Last update [4e817ff...23041d2](https://codecov.io/gh/huggingface/transformers/pull/4000?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>There's no option to do `--do_lower_case` in the scripts anymore, the issue is that this particular tokenizer should have a `tokenizer_config.json` that always applies lowercasing.<|||||>The option `--do_lower_case` seems to be available in the [run_squad.py](https://github.com/huggingface/transformers/blob/master/examples/run_squad.py#L579-L581), currently (but not applied in the official readme).
It's my understanding that `--do_lower_case` option should be applied, in order to reproduce the official results described in readme (only uncased models' examples).

Or do you mean that `--do_lower_case` should not be used in the current (or future) version?<|||||>Yes, it shouldn’t be used in current or future versions of the scripts (because it’s handled in tokenizer configuration)<|||||>In your case, you should apply it manually for now, while we fix the root issue<|||||>Thank you for your answers!
I understand the issue and its solution, and so I will be closing this pull request.