lowercasing on LM with cased models
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Execute 'run_language_modeling.py' with one of the following cased models.
   (example of cased models: sciBert cased, BioBert, ClinicalBert)
2. Check the tokenizer's do_lower_case option.

## Expected behavior

I'm language modeling with my own data on the top of a pre-trained model. I'm using cased models, but the tokenizer lower case input data since its default lowercase option is True. When I used 'bert-base-cased', the tokenizer didn't lower case, but it happened with other cased models mentioned above. 
- tokens with 'bert-base-cased' model
['[CLS]', 'This', 'text', 'is', 'included', 'to', 'make', 'sure', 'Uni', '##code',...
- tokens with 'scibert_scivocab_cased' model
['[CLS]', 'this', 'text', 'is', 'included', 'to', 'make', 'sure', 'unic', '##ode',...

Is it a bug? or am I missing something?

As an alternative, I'm using a workaround code by adding additional command parameter.
```python
parser.add_argument("--do_lower_case", action="store_true", help="Should be added for uncased models.")

tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path, cache_dir=args.cache_dir, do_lower_case=args.do_lower_case)
```



Thanks in advance.

Unfortunately, this is because these models didn't get uploaded with the `tokenizer_config.json` specifying they shouldn't be lowercased.

cc @julien-c <|||||>I see. Thanks for the explanation. I think it would be helpful to mention that in the instruction or somewhere for newbies like me though. They may overlook it unless they actually test a model before training it.<|||||>Yes, if those models really are lowercase, we should add a `tokenizer_config.json` (and let their authors/uploaders know about it). Also cc @patrickvonplaten <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
