 Model Description  This model was captured during a reproduction of [RoBERTa-base](https://huggingface.co/roberta-base), for English: it is a Transformers model pretrained on a large corpus of English data, using the Masked Language Modelling (MLM).  The intended uses, limitations, training data and training procedure for the fully trained model are similar to [RoBERTa-base](https://huggingface.co/roberta-base). Two major differences with the original model:  *   We trained our model for 100K steps, instead of 500K *   We only use Wikipedia and the Book Corpus, as corpora which are publicly available.   