 Model description  BERTa is a transformer-based masked language model for the Catalan language.  It is based on the [RoBERTA](https://github.com/pytorch/fairseq/tree/master/examples/roberta) base model  and has been trained on a medium-size corpus collected from publicly available corpora and crawlers.  