Do Large Language Models Learn Human-Like Strategic Preferences?
<|reference_start|>Do Large Language Models Learn Human-Like Strategic Preferences?: In this paper, we evaluate whether LLMs learn to make human-like preference judgements in strategic scenarios as compared with known empirical results. Solar and Mistral are shown to exhibit stable value-based preference consistent with humans and exhibit human-like preference for cooperation in the prisoner's dilemma (including stake-size effect) and traveler's dilemma (including penalty-size effect). We establish a relationship between model size, value-based preference, and superficiality. Finally, results here show that models tending to be less brittle have relied on sliding window attention suggesting a potential link. Additionally, we contribute a novel method for constructing preference relations from arbitrary LLMs and support for a hypothesis regarding human behavior in the traveler's dilemma.<|reference_end|>
@article{roberts2024do,
    title={Do Large Language Models Learn Human-Like Strategic Preferences?},
    author={Jesse Roberts, Kyle Moore, Doug Fisher},
    journal={arXiv preprint arXiv:2404.08710},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.08710},
    primaryClass={cs.GT cs.AI}
}