Training mask language model using multiple files
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.5
- Platform:Linux
- Python version:3.7
- PyTorch version (GPU?):1.7.1
- Tensorflow version (GPU?):
- Using GPU in script?:Yes
- Using distributed or parallel set-up in script?:No

Hi all,

I am very new to transformer. Now I am training my own mask language models using big data (~500G). And these data are saved in several files. When I follow the tutorial (https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb#scrollTo=YpvnFFmZJD-N), I find that they always train the model using only one training file. So, my question is that how to train the model using several training files? Hope you can help me! Thank you very very much!

@lindan1128 the same code can be used for multiple files too, check this cell:-

```
%%time 
from pathlib import Path

from tokenizers import ByteLevelBPETokenizer

paths = [str(x) for x in Path(".").glob("**/*.txt")]

# Initialize a tokenizer
tokenizer = ByteLevelBPETokenizer()

# Customize training
tokenizer.train(files=paths, vocab_size=52_000, min_frequency=2, special_tokens=[
    "<s>",
    "<pad>",
    "</s>",
    "<unk>",
    "<mask>",
])
```

Here paths will get all the txt files in the current directory and will read all of them.<|||||>@frankhart2018 It indeed works!!! Thank you very much!!!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.