XNLIeu: a dataset for cross-lingual NLI in Basque
<|reference_start|>XNLIeu: a dataset for cross-lingual NLI in Basque: XNLI is a popular Natural Language Inference (NLI) benchmark widely used to evaluate cross-lingual Natural Language Understanding (NLU) capabilities across languages. In this paper, we expand XNLI to include Basque, a low-resource language that can greatly benefit from transfer-learning approaches. The new dataset, dubbed XNLIeu, has been developed by first machine-translating the English XNLI corpus into Basque, followed by a manual post-edition step. We have conducted a series of experiments using mono- and multilingual LLMs to assess a) the effect of professional post-edition on the MT system; b) the best cross-lingual strategy for NLI in Basque; and c) whether the choice of the best cross-lingual strategy is influenced by the fact that the dataset is built by translation. The results show that post-edition is necessary and that the translate-train cross-lingual strategy obtains better results overall, although the gain is lower when tested in a dataset that has been built natively from scratch. Our code and datasets are publicly available under open licenses.<|reference_end|>
@article{heredia2024xnlieu:,
    title={XNLIeu: a dataset for cross-lingual NLI in Basque},
    author={Maite Heredia, Julen Etxaniz, Muitze Zulaika, Xabier Saralegi, Jeremy
  Barnes, Aitor Soroa},
    journal={arXiv preprint arXiv:2404.06996},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.06996},
    primaryClass={cs.CL cs.AI}
}