Hello there
I am trying to run the script run_mlm.py to perform the training of a BERT model. Basically, the idea is to start from an already existing italian BERT model, perform a second training on a specific topic (biomedical texts), and later fine-tune it on a QuestionAnswering downstream task.
I was able to run the run_mlm.py script, both with and without –line_by_line parameter. I have a couple of questions, if you could kindly answer or point me to somewhere in the documentation:


The run with –line_by_line took like 10x longer that the one without, why? I have full access to the complete dataset, and I can organize it as I want, so which is the best format?


Is there a way to feed the model more files, if my corpus is split on several of them?


Does this script train the model for the NSP task as well?


If I evaluate the model, I get the perplexity score. Is there a way to get accuracy for the NSP task? (I think that accuracy does not make sense for MLM, right?)


Many thanks for your patience