 Benchmark performance  We tested the performance of **XLMR-MaCoCu-is** on benchmarks of XPOS, UPOS, NER and COPA. For UPOS and XPOS, we used the data from the [Universal Dependencies](https://universaldependencies.org/) project. For NER, we used the data from the MIM-GOLD-NER data set. For COPA, we automatically translated the English data set by using Google Translate. For details please see our [Github repo](https://github.com/RikVN/COPA). We compare performance to the strong multi-lingual models XLMR-base and XLMR-large, but also the monolingual [IceBERT](https://huggingface.co/vesteinn/IceBERT) model. For details regarding the XPOS/UPOS/NER fine-tuning procedure you can checkout our [Github](https://github.com/macocu/LanguageModels).  Scores are averages of three runs, except for COPA, for which we use 10 runs. We use the same hyperparameter settings for all models.  |                    | **UPOS** | **UPOS** | **XPOS** | **XPOS** | **NER** | **NER**  | **COPA**  | |--------------------|:--------:|:--------:|:--------:|:--------:|---------|----------| ----------| |                    |  **Dev** | **Test** |  **Dev** | **Test** | **Dev** | **Test** | **Test** | | **XLM-R-base**     |   96.8   |   96.5   |   94.6   |   94.3   |   85.3  |   89.7   | 55.2 | | **XLM-R-large**    |   97.0   |   96.7   |   94.9   |   94.7   |   88.5  |   91.7   | 54.3 | | **IceBERT**        |   96.4   |   96.0   |   94.0   |   93.7   |   83.8  |   89.7   | 54.6 |  | **XLMR-MaCoCu-is** |   **97.3**   |    **97.0**    |   **95.4**   |   **95.1**  |   **90.8**  |   **93.2**   | **59.6** |  