readme: ---
  language: 
  - et
  
  license: cc-by-sa-4.0
  ---
  # Usage
  Load in transformers library with:
  ```
  from transformers import AutoTokenizer, AutoModelForMaskedLM
    
    tokenizer = AutoTokenizer.from_pretrained("EMBEDDIA/est-roberta")
    model = AutoModelForMaskedLM.from_pretrained("EMBEDDIA/est-roberta")
  ```
  
  # Est-RoBERTa
  Est-RoBERTa model is a monolingual Estonian BERT-like model. It is closely related to French Camembert model https://camembert-model.fr/. The Estonian corpora used for training the model have 2.51 billion tokens in total. The subword vocabulary contains 40,000 tokens.
  
  Est-RoBERTa was trained for 40 epochs.
  
  