Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
Human readers can efficiently comprehend scrambled words, a phenomenon known
as Typoglycemia, primarily by relying on word form; if word form alone is
insufficient, they further utilize contextual cues for interpretation. While
advanced large language models (LLMs) exhibit similar abilities, the underlying
mechanisms remain unclear. To investigate this, we conduct controlled
experiments to analyze the roles of word form and contextual information in
semantic reconstruction and examine LLM attention patterns. Specifically, we
first propose SemRecScore, a reliable metric to quantify the degree of semantic
reconstruction, and validate its effectiveness. Using this metric, we study how
word form and contextual information influence LLMs' semantic reconstruction
ability, identifying word form as the core factor in this process. Furthermore,
we analyze how LLMs utilize word form and find that they rely on specialized
attention heads to extract and process word form information, with this
mechanism remaining stable across varying levels of word scrambling. This
distinction between LLMs' fixed attention patterns primarily focused on word
form and human readers' adaptive strategy in balancing word form and contextual
information provides insights into enhancing LLM performance by incorporating
human-like, context-aware mechanisms.
67c6d22d983375492193aab0
Chenxi Wang
claimed_verified
2025-03-04T11:16:44.551Z
679bc0ec7f3c28bf968321c8
https://cdn-avatars.huggingface.co/v1/production/uploads/679bc0ec7f3c28bf968321c8/uFqCcPbVjDISww46Jbovf.jpeg
Chenxi Wang
user
Aurora-cx
67c6d22d983375492193aab1
Tianle Gu
admin_assigned
2025-03-04T10:54:29.015Z
6346361c5efccdc07f179cae
/avatars/217818114a4c19ea4f3e5cdafefb625e.svg
Gu Tianle
user
Carol0110
67c6d22d983375492193aab2
Zhongyu Wei
67c6d22d983375492193aab3
Lang Gao
67c6d22d983375492193aab4
Zirui Song
claimed_verified
2025-03-04T10:17:25.935Z
65407ba7a38390065750233f
https://cdn-avatars.huggingface.co/v1/production/uploads/65407ba7a38390065750233f/1_IPMZbk-S9u2t18PQgMp.jpeg
Zirui Song
user
Ziruibest
67c6d22d983375492193aab5
Xiuying Chen
Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
Human readers can efficiently comprehend scrambled words, a phenomenon known
as Typoglycemia, primarily by relying on word form; if word form alone is
insufficient, they further utilize contextual cues for interpretation. While
advanced large language models (LLMs) exhibit similar abilities, the underlying
mechanisms remain unclear. To investigate this, we conduct controlled
experiments to analyze the roles of word form and contextual information in
semantic reconstruction and examine LLM attention patterns. Specifically, we
first propose SemRecScore, a reliable metric to quantify the degree of semantic
reconstruction, and validate its effectiveness. Using this metric, we study how
word form and contextual information influence LLMs' semantic reconstruction
ability, identifying word form as the core factor in this process. Furthermore,
we analyze how LLMs utilize word form and find that they rely on specialized
attention heads to extract and process word form information, with this
mechanism remaining stable across varying levels of word scrambling. This
distinction between LLMs' fixed attention patterns primarily focused on word
form and human readers' adaptive strategy in balancing word form and contextual
information provides insights into enhancing LLM performance by incorporating
human-like, context-aware mechanisms.
LLMs rely on word form for semantic reconstruction and utilize specialized attention heads, differing from human readers who balance word form and context adaptively.
advanced large language models LLMs semantic reconstruction SemRecScore attention patterns attention heads human readers