Context Limitations Make Neural Language Models More Human-Like
Language models (LMs) have been used in cognitive modeling as well as engineering studies---they compute information-theoretic complexity metrics that simulate humans′ cognitive load during reading.This study highlights a limitation of modern neural LMs as the model of choice for this purpose: there is a discrepancy between their context access capacities and that of humans.Our results showed that constraining the LMs′ context access improved their simulation of human reading behavior.We also showed that LM-human gaps in context access were associated with specific syntactic constructions; incorporating syntactic biases into LMs′ context access might enhance their cognitive plausibility.