111:24

Trovato and Tobin, et al.

focuses on assessing language models’ comprehension and utilization of semantic knowledge
for inference. As such, KoLA serves as an important benchmark for evaluating the depth of lan-
guage understanding and reasoning in language models, thereby driving progress in language
comprehension. To enable crowd-sourced evaluations of language tasks, DynaBench [87] supports
dynamic benchmark testing. DynaBench explores new research directions including the effects of
closed-loop integration, distributional shift characteristics, annotator efficiency, influence of expert
annotators, and model robustness to adversarial attacks in interactive settings. Furthermore, to
evaluate language models’ ability to learn and apply multidisciplinary knowledge across educational
levels, the Multidisciplinary Knowledge Evaluation M3KE [113] was recently introduced. M3KE
assesses knowledge application within the Chinese education system.
ASurveyonEvaluationofLargeLanguageModels