6.2 Evaluation for LLM-based Agents

While LLM-based agents have demonstrated excellent performance in areas such as standalone
operation, collective cooperation, and human interaction, quantifying and objectively evaluating
them remains a challenge [582; 89]. Turing proposed a highly meaningful and promising approach
for assessing AI agents—the well-known Turing Test—to evaluate whether AI systems can exhibit
human-like intelligence [3]. However, this test is exceedingly vague, general, and subjective. Here,
we discuss existing evaluation efforts for LLM-based agents and offer some prospects, considering
four dimensions: utility, sociability, values, and the ability to evolve continually.
TheRiseandPotentialofLargeLanguageModel BasedAgents