6.5 Results
Our findings suggest that the full architecture of generative agents
generates the most believable behavior among all study conditions.
We contrast the full architecture’s responses to those of other con-
ditions below. However, we also report that the full architecture
was not without flaws and illustrate its modes of failures.
6.5.1 The full architecture bests other conditions. As seen in Fig-
ure 8, the full generative agent architecture produced the most
believable behavior (𝜇 = 29.89; 𝜎 = 0.72). Performance degraded
with the removal of each component in the ablation conditions:
the ablated architecture with no access to reflection was the next
best (𝜇 = 26.88; 𝜎 = 0.69), followed by no access to reflection or
planning (𝜇 = 25.64; 𝜎 = 0.68), followed by the human condition
(𝜇 = 22.95; 𝜎 = 0.69). The ablated architecture with no access to
memory, planning, or reflection performed the worst of all con-
ditions (𝜇 = 21.21; 𝜎 = 0.70). TrueSkill models each condition’s
Generative Agents- Interactive Simulacra of Human Behavior