Instead of focusing on specific tasks and evaluation metrics, HELM [107] provides a comprehen-
sive assessment of LLMs. It evaluates language models across various aspects such as language
understanding, generation, coherence, context sensitivity, common-sense reasoning, and domain-
specific knowledge. HELM aims to holistically evaluate the performance of language models across
different tasks and domains. For LLMs Evaluator, Zhang et al. [238] introduces LLMEval2, which en-
compasses a wide range of capability evaluations. In addition, Xiezhi [55] presents a comprehensive
suite for assessing the knowledge level of large-scale language models in different subject areas. The
evaluation conducted through Xiezhi enables researchers to comprehend the notable limitations
inherent in these models and facilitates a deeper comprehension of their capabilities in diverse
fields. For evaluating language models beyond their existing capacities, BIG-bench [172] introduces
ASurveyonEvaluationofLargeLanguageModels