LM fine tuning on top of a custom model
Currently the `finetune_on_pregenerated.py` script allows only fine-tuning on top of one of the five pretrained bert models. However I don't understand why there is such a restriction. I am trying to finetune a LM on top of a custom bert model (mt-dnn). Of course I can just remove the `choices`, but I am wondering if there is some rationale behind this, as none of the other example scripts contains this restriction.

https://github.com/huggingface/pytorch-pretrained-BERT/blob/94980b529fad34f55c5d34c94bae2814db6773a6/examples/lm_finetuning/finetune_on_pregenerated.py#L126-L128
Not really, I guess we could remove this restriction and just let people point to an arbitrary folder (as we do in the other scripts).
What do you think @Rocketknight1?<|||||>No, that's entirely my bad. We should allow an arbitrary folder!<|||||>Should I just remove the `choices` argument and allow any string there?<|||||>Yes, I would go for that!<|||||>Actually, if you have push access to the repo, do you want to just make that one-line change? I'd have to fork and submit a PR, which seems a bit unnecessary.<|||||>Indeed, I'll take care of it. I'm on the repo now anyway.<|||||>is this change applicable and working with run_lm_finetuning.py in 2.x?