Not able to train RoBERTa language model from scratch
I tried training RoBERTa language model from scratch using 

<code> !python ./run_mlm.py 
    --model_name_or_path roberta-base 
    --train_file './data_lm.txt' 
    --do_train 
    --line_by_line 
    --num_train_epochs 3
    --output_dir ./roberta
</code>

Due to the limits on the colab storage, I deleted all the checkpoints generated during the training process. So finally my model directory has the following files - 
1. config.json
2. merges.txt
3. pytorch_model.bin
4. special_tokens_map.json
5. tokenizer.config
6. vocab.json

But after loading my trained model using RobertaTokenizer, RobertaForSequenceClassification and fine-tuning it  for my classification task, i am receiving almost same accuracy as by loading and fine-tuning the readily available 'roberta-base'. 

Also, when i try loading it as
`model = RobertaModel.from_pretrained('./roberta)
`

 I get the warning-

<code>
Some weights of RobertaModel were not initialized from the model checkpoint at ./roberta/ and are newly initialized: ['roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
</code>

So, my question is, Is there something wrong in the training procedure of the Language Model on my dataset and the loading process? Or the checkpoints that i deleted were important?  
 
The `run_mlm` script trains a `RobertaForMaskedLM` model, which does not have a pooler layer. That's why you get this warning when using this pretrained model to initialize a `RobertaModel`.<|||||>Thanks @sgugger! Now I get where the problem was. 
Moreover, I found some good tutorials on it.  Here are the links for those who need it.
https://zablo.net/blog/post/training-roberta-from-scratch-the-missing-guide-polish-language-model/
https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb