Recent work has led to the development of benchmarks for evaluating language models’ knowl-
edge and reasoning abilities. The Knowledge-Oriented Language Model Evaluation KoLA [223]

5Note that as the evaluation of LLMs is a hot research area, it is very likely that we cannot cover all benchmarks. We
welcome suggestions and comments to make this list perfect.

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

111:24

Trovato and Tobin, et al.
ASurveyonEvaluationofLargeLanguageModels