readme: ---
  language:
  - uk
  tags:
  - ukrainian
  widget:
   - text: "Тарас Шевченко – великий український <mask>."
  license: mit
  ---
  This is a smaller version of the [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) model with only Ukrainian and some English embeddings left. 
  
  * The original model has 470M parameters, with 384M of them being input and output embeddings. 
  * After shrinking the `sentencepiece` vocabulary from 250K to 31K (top 25K Ukrainian tokens and top English tokens) the number of model parameters reduced to 134M parameters, and model size reduced from 1GB to 400MB.
  