Finetuning distilbert-base-uncased
## ❓ Questions & Help
When trying to finetune distilbert-base-uncased on my own dataset  
I receive the following error message:
ERROR - pytorch_transformers.tokenization_utils -   Model name 'distilbert-base-uncased' was not found in model name list (bert-base-uncased,bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc). 

How do I add distilbert-base-uncased to the model name list (I assume I will somehow have to modify PYTORCH-TRANSFORMERS_CACHE)




code used:
python simple_lm_finetuning.py \
--train_corpus dataset.txt \
--bert_model distilbert-base-uncased \
--do_lower_case \
--output_dir finetuned_lm/ \
--do_train
Hi! Yes we're in the process of adding DistilBERT to the examples. Until then, you can simply edit the script to add it.

Please note that the `simple_lm_finetuning` script is now deprecated in favor of `run_lm_finetuning`.<|||||>Hi there @aah39 , I came across the same issue in run_glue.py when I tried to fine tune distilbert_base_uncased. Later I found the fix was easy: just change the model_type to be distilbert when running the script (I saw run_lm_finetuning has this input parameter as well)
After this change, when running the script, it will load the cache from DISTILBERT_PRETRAINED_CONFIG_ARCHIVE_MAP instead of BERT_PRETRAINED_CONFIG_ARCHIVE_MAP<|||||>I think this is not an issue anymore since it's been fixed in the https://github.com/huggingface/transformers/commit/88368c2a16d26bc2d00dc28f79196c81373d3a71
