🇪🇪 Estonian LLM Evaluation
A collection of resources for evaluation of LLM capabilities in the Estonian language.
Viewer • Updated • 541 • 173Note Manually translated and culturally adapted IFEval.
tartuNLP/winogrande_et
Viewer • Updated • 4.83k • 190Note Manually translated and culturally adapted WinoGrande test set.
TalTechNLP/EstQA
Viewer • Updated • 1.38k • 81 • 2Note Manually annotated extractive question answering.
tartuNLP/EstNER
Viewer • Updated • 46.1k • 31Note Manually annotated named entity recognition.
tartuNLP/EstCOPA
Viewer • Updated • 2k • 12 • 1Note Machine-translated and post-edited choice of plausible alternatives.
TalTechNLP/grammar_et
Viewer • Updated • 8.94k • 17 • 1Note Grammatical error correction. Text with errors → corrected text.
TalTechNLP/ERRnews
Updated • 58 • 1Note ERR news story transcripts generated by an ASR pipeline paired with the human written summary.
kardosdrur/estonian-valence
Viewer • Updated • 4.09k • 450 • 1Note Newspaper article sentiment.
TalTechNLP/exam_et
Viewer • Updated • 1.61k • 19Note Exam questions on several subjects in multiple choice format.
TalTechNLP/MMLU_et
Viewer • Updated • 14k • 124 • 1Note Machine-translated MMLU.
TalTechNLP/word_meanings_et
Viewer • Updated • 54.6k • 6Note Matching words with their dictionary definitions.
TalTechNLP/inflection_et
Viewer • Updated • 1.4k • 5 • 1Note Noun phrase inflection.
TalTechNLP/trivia_et
Viewer • Updated • 800 • 32 • 1Note Trivia dataset extracted from the "Eesti Mäng" board game.
TalTechNLP/MMLU_Pro_et
Viewer • Updated • 12.1k • 3Note Machine-translated MMLU-Pro.
TalTechNLP/M_MMLU_et
Viewer • Updated • 15k • 5
tartuNLP/smugri-mt-bench
Viewer • Updated • 500 • 92 • 2Note Extension of the MT-Bench dataset including Estonian, Livonian, Komi, and Võro.
tartuNLP/flores-smugri-pairs
Viewer • Updated • 16.5k • 132Note Extension of the original Flores dataset adding additional translation directions, such as Estonian to Võro, Estonian to Komi-Zyrian, and some others.
facebook/belebele
Viewer • Updated • 110k • 14.5k • 114Note Multiple choice reading comprehension. Includes Estonian as one of the languages.
cambridgeltl/xcopa
Viewer • Updated • 12.6k • 6.49k • 18Note Machine-translated multilingual COPA. Includes validation and test for Estonian.
jumelet/multiblimp
Viewer • Updated • 121k • 720 • 12Note Linguistic acceptability dataset derived from Universal Dependencies with heuristics. Includes Estonian as one of the languages.
openlanguagedata/flores_plus
Viewer • Updated • 872k • 5.22k • 62Note Includes Estonian as one of the languages. English sources are translated to 200 languages. Any translation direction is possible potentially.
google/wmt24pp
Viewer • Updated • 54.9k • 3.91k • 60Note Includes English to Estonian translation direction.
mteb/sib200
Viewer • Updated • 387k • 2.49k • 3Note Includes Estonian as one of the languages, classification labels remain in English.