CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations
<|reference_start|>CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations: In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities.<|reference_end|>
@article{zhao2024cpsyexam:,
    title={CPsyExam: A Chinese Benchmark for Evaluating Psychology using
  Examinations},
    author={Jiahao Zhao, Jingwei Zhu, Minghuan Tan, Min Yang, Di Yang, Chenhao
  Zhang, Guancheng Ye, Chengming Li, Xiping Hu},
    journal={arXiv preprint arXiv:2405.10212},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.10212},
    primaryClass={cs.CL}
}