GPT-4 Understands Discourse at Least as Well as Humans Do
<|reference_start|>GPT-4 Understands Discourse at Least as Well as Humans Do: We test whether a leading AI system GPT-4 understands discourse as well as humans do, using a standardized test of discourse comprehension. Participants are presented with brief stories and then answer eight yes/no questions probing their comprehension of the story. The questions are formatted to assess the separate impacts of directness (stated vs. implied) and salience (main idea vs. details). GPT-4 performs slightly, but not statistically significantly, better than humans given the very high level of human performance. Both GPT-4 and humans exhibit a strong ability to make inferences about information that is not explicitly stated in a story, a critical test of understanding.<|reference_end|>
@article{shultz2024gpt-4,
    title={GPT-4 Understands Discourse at Least as Well as Humans Do},
    author={Thomas Shultz, Jamie Wise, Ardavan Salehi Nobandegani},
    journal={arXiv preprint arXiv:2403.17196},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.17196},
    primaryClass={cs.CL}
}