train model from scratch with big data
# 🚀 Feature request
<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

I would like to train a new model from scratch with some big data sets and the tutorial suggests to load and tokenize examples on the fly. It would be great if that feature is readily integrated in the example scripts (e.g. `run_language_modeling.py`). It is not entirely clear to me how to implement this in the most efficient way. My data set does not fit into memory and I cannot train out-of-the box with the existing scripts.

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
