From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for
  LLM Evaluation
The development of Large Language Models (LLMs) requires robust benchmarks
that encompass not only academic domains but also industrial fields to
effectively evaluate their applicability in real-world scenarios. In this
paper, we introduce two Korean expert-level benchmarks. KMMLU-Redux,
reconstructed from the existing KMMLU, consists of questions from the Korean
National Technical Qualification exams, with critical errors removed to enhance
reliability. KMMLU-Pro is based on Korean National Professional Licensure exams
to reflect professional knowledge in Korea. Our experiments demonstrate that
these benchmarks comprehensively represent industrial knowledge in Korea. We
release our dataset publicly available.
6875aa3c257d4f043537052c
Seokhee Hong
6875aa3c257d4f043537052d
Sunkyoung Kim
claimed_verified
2025-07-16T15:06:42.970Z
628efeff6d8dcc7ab5263881
/avatars/1e894ccd7ba206a755b0b9af9f22ead1.svg
Sunkyoung Kim
user
Sunkyoung
6875aa3c257d4f043537052e
Guijin Son
6875aa3c257d4f043537052f
Soyeon Kim
6875aa3c257d4f0435370530
Yeonjung Hong
6875aa3c257d4f0435370531
Jinsik Lee
From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for
  LLM Evaluation
The development of Large Language Models (LLMs) requires robust benchmarks
that encompass not only academic domains but also industrial fields to
effectively evaluate their applicability in real-world scenarios. In this
paper, we introduce two Korean expert-level benchmarks. KMMLU-Redux,
reconstructed from the existing KMMLU, consists of questions from the Korean
National Technical Qualification exams, with critical errors removed to enhance
reliability. KMMLU-Pro is based on Korean National Professional Licensure exams
to reflect professional knowledge in Korea. Our experiments demonstrate that
these benchmarks comprehensively represent industrial knowledge in Korea. We
release our dataset publicly available.
Two Korean expert-level benchmarks, KMMLU-Redux and KMMLU-Pro, are introduced to evaluate Large Language Models in industrial and professional contexts.
Large Language Models benchmarks KMMLU-Redux KMMLU-Pro Korean National Technical Qualification exams Korean National Professional Licensure exams