KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinations
<|reference_start|>KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinations: We introduce KorMedMCQA, the first Korean multiple-choice question answering (MCQA) benchmark derived from Korean healthcare professional licensing examinations, covering from the year 2012 to year 2023. This dataset consists of a selection of questions from the license examinations for doctors, nurses, and pharmacists, featuring a diverse array of subjects. We conduct baseline experiments on various large language models, including proprietary/open-source, multilingual/Korean-additional pretrained, and clinical context pretrained models, highlighting the potential for further enhancements. We make our data publicly available on HuggingFace (https://huggingface.co/datasets/sean0042/KorMedMCQA) and provide a evaluation script via LM-Harness, inviting further exploration and advancement in Korean healthcare environments.<|reference_end|>
@article{kweon2024kormedmcqa:,
    title={KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean
  Healthcare Professional Licensing Examinations},
    author={Sunjun Kweon, Byungjin Choi, Minkyu Kim, Rae Woong Park, Edward Choi},
    journal={arXiv preprint arXiv:2403.01469},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.01469},
    primaryClass={cs.CL}
}