In another study by Hagendorff and Fabi [57], LLMs with enhanced cognitive abilities were
evaluated. They found that these models can avoid common human intuitions and cognitive errors,
demonstrating super-rational performance. By utilizing cognitive reflection tests and semantic
illusion experiments, the researchers gained insights into the psychological aspects of LLMs. This

3The term ‘trustworthiness’ in this section refers to other work that contains more than robustness and ethics.

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

A Survey on Evaluation of Large Language Models

111:15

method offers new perspectives for evaluating model biases and ethical issues that may not have
been previously identified.
ASurveyonEvaluationofLargeLanguageModels