Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function
<|reference_start|>Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function: What makes large language models (LLMs) impressive is also what makes them hard to evaluate: their diversity of uses. To evaluate these models, we must understand the purposes they will be used for. We consider a setting where these deployment decisions are made by people, and in particular, people's beliefs about where an LLM will perform well. We model such beliefs as the consequence of a human generalization function: having seen what an LLM gets right or wrong, people generalize to where else it might succeed. We collect a dataset of 19K examples of how humans make generalizations across 79 tasks from the MMLU and BIG-Bench benchmarks. We show that the human generalization function can be predicted using NLP methods: people have consistent structured ways to generalize. We then evaluate LLM alignment with the human generalization function. Our results show that -- especially for cases where the cost of mistakes is high -- more capable models (e.g. GPT-4) can do worse on the instances people choose to use them for, exactly because they are not aligned with the human generalization function.<|reference_end|>
@article{vafa2024do,
    title={Do Large Language Models Perform the Way People Expect? Measuring the
  Human Generalization Function},
    author={Keyon Vafa, Ashesh Rambachan, Sendhil Mullainathan},
    journal={arXiv preprint arXiv:2406.01382},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.01382},
    primaryClass={cs.CL cs.AI}
}