run_lm_finetuning on multiple training files
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->

Hi,

I would like to fine-tune huggingface's pre-trained BERT model on a relatively big text data.
I split the data into multiple files (~10k files) in .raw format in the same folder.

First, I succeeded to run run_lm_finetuning.py on one of the raw files I generated.

Now, I would like to run that in my 10k raw files.
I realised that that train_data_file argument does not accept a folder name (only the path to a single raw file).
I believe that we can do a loop over the 10k train files and incrementally fine-tuning the model, but it does not look like a best solution for me...

Could you please tell me if it exists a simple way to achieve the above?

Thank you very much for your help.
Hi, this is not currently supported, you would need to implement this yourself. Feel free to open a PR if you do