Evaluating Multimodal Generative AI with Korean Educational Standards
This paper presents the Korean National Educational Test Benchmark (KoNET), a
new benchmark designed to evaluate Multimodal Generative AI Systems using
Korean national educational tests. KoNET comprises four exams: the Korean
Elementary General Educational Development Test (KoEGED), Middle (KoMGED), High
(KoHGED), and College Scholastic Ability Test (KoCSAT). These exams are
renowned for their rigorous standards and diverse questions, facilitating a
comprehensive analysis of AI performance across different educational levels.
By focusing on Korean, KoNET provides insights into model performance in
less-explored languages. We assess a range of models - open-source,
open-access, and closed APIs - by examining difficulties, subject diversity,
and human error rates. The code and dataset builder will be made fully
open-sourced at https://github.com/naver-ai/KoNET.
67bc53ad670ece8d919a8fe1
Sanghee Park
claimed_verified
2025-02-24T11:54:51.633Z
6639f75a910d619c288f8a86
/avatars/4b20a056798c009eaf665b0e3021db60.svg
sanghee park
user
sangheeeee
67bc53ad670ece8d919a8fe2
Geewook Kim
claimed_verified
2025-02-24T13:04:57.334Z
6298362c9d3de7b32fd11526
https://cdn-avatars.huggingface.co/v1/production/uploads/1658473855720-6298362c9d3de7b32fd11526.jpeg
Geewook Kim
user
gwkrsrch
Evaluating Multimodal Generative AI with Korean Educational Standards
This paper presents the Korean National Educational Test Benchmark (KoNET), a
new benchmark designed to evaluate Multimodal Generative AI Systems using
Korean national educational tests. KoNET comprises four exams: the Korean
Elementary General Educational Development Test (KoEGED), Middle (KoMGED), High
(KoHGED), and College Scholastic Ability Test (KoCSAT). These exams are
renowned for their rigorous standards and diverse questions, facilitating a
comprehensive analysis of AI performance across different educational levels.
By focusing on Korean, KoNET provides insights into model performance in
less-explored languages. We assess a range of models - open-source,
open-access, and closed APIs - by examining difficulties, subject diversity,
and human error rates. The code and dataset builder will be made fully
open-sourced at https://github.com/naver-ai/KoNET.
KoNET evaluates Multimodal Generative AI Systems using Korean national educational tests across various levels, focusing on language-specific performance.
Multimodal Generative AI Systems KoNET KoEGED KoMGED KoHGED KoCSAT