3.2 Robustness, Ethic, Bias, and Trustworthiness
The evaluation of LLMs encompasses the crucial aspects of robustness, ethics, biases, and trustwor-
thiness. These factors have gained increasing importance in assessing the performance of LLMs
comprehensively.
3.2.1 Robustness. Robustness studies the stability of a system when facing unexpected inputs.
Specifically, out-of-distribution (OOD) [195] and adversarial robustness are two popular research
topics for robustness. Wang et al. [194] is an early work that evaluated ChatGPT and other LLMs
from both the adversarial and OOD perspectives using existing benchmarks such as AdvGLUE [191],
ANLI [131], and DDXPlus [38] datasets. Zhuo et al. [252] evaluated the robustness of semantic
parsing. Yang et al. [221] evaluated OOD robustness by extending the GLUE [188] dataset. The
results of this study emphasize the potential risks to the overall system security when manipulating
ASurveyonEvaluationofLargeLanguageModels