3

LLMs [40], nor evaluate the psychometric properties of the assessments [41]. In con-
trast, the presented work establishes psychometrics-grounded construct validity when
administering personality inventories to LLMs, thus creating a scientific foundation
for quantifying personality in LLMs. To claim that simulated psychological test scores
are meaningful in comparing LLM and human behavior, psychometrics requires estab-
lishing the construct validity of these simulated tests in terms of their 1) structural
validity, 2) convergent and discriminant validity, and 3) criterion validity. Our work
addresses these rigorous requirements which are lacking in previous efforts.
PersonalityTraitsinLargeLanguageModels