Description of how to preprocess text corpus for roBERTa LM training
# 🚀 Feature request

There are different locations where you desribe how to train roBERTa models. For example here:
- https://huggingface.co/blog/how-to-train
- https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb
- https://gist.github.com/aditya-malte/2d4f896f471be9c38eb4d723a710768b

But nobody sais how to preprocess the text corpus. I think it must be one sentence per row. But does it need empty lines between documents? Is it ok to shuffle the text line by line?

Could you please claify this?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Well - this issue is still open IMO...<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Maybe better to ask to the original RoBERTa authors or on the forum at https://discuss.huggingface.co?

We are trying to keep the issues for bug/features reports now.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
