pre-training roberta from scratch
Hi,

I am currently pre-training RoBERTa model from scratch based on the tutorial in here: https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb and I got 3 questions.

1) I see that the data used in this tutorial is separated line by line and the train and test data is put into LineByLineTextDataset function before training. What is the usage of this function ? I couldn't find an explanation of it in Huggingface documents. Even though I have guesses, I would like to know better.  As I see the data in the tutorial is already split line by line. So, what would have been lost if these data are not put LineByLineTextDataset function before starting the training process ? 

2) The data I will use in pre-training process is not split line by line. In my data, each example have multiple sentences. So, I assume that I shouldn't use LineByLineTextDataset function. Is there any other function I can use instead ? Or would it be wrong if I would join the whole dataset in one very long text and then split it into sentences and then use LineByLineTextDataset function ?

3) This will be a little bit technical question. In RoBERTa paper, it was written that the size of the data they used for pre-training process is 160 GB.  If I would pre-train the model with 10-20 GB of data, would this decrease the performance of the pre-trained model drastically or is it okay to use as long as the training arguments are set accordingly ? 
Hi @ozyurtf !

Thank you for opening the issue. It would be nice if you could ask such general questions on the [forum](https://discuss.huggingface.co/). We use issues for bug reports and feature requests.<|||||>Okay, I will. Thank you Suraj!