title: Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
summary: Human readers can efficiently comprehend scrambled words, a phenomenon known
  as Typoglycemia, primarily by relying on word form; if word form alone is
  insufficient, they further utilize contextual cues for interpretation. While
  advanced large language models (LLMs) exhibit similar abilities, the underlying
  mechanisms remain unclear. To investigate this, we conduct controlled
  experiments to analyze the roles of word form and contextual information in
  semantic reconstruction and examine LLM attention patterns. Specifically, we
  first propose SemRecScore, a reliable metric to quantify the degree of semantic
  reconstruction, and validate its effectiveness. Using this metric, we study how
  word form and contextual information influence LLMs' semantic reconstruction
  ability, identifying word form as the core factor in this process. Furthermore,
  we analyze how LLMs utilize word form and find that they rely on specialized
  attention heads to extract and process word form information, with this
  mechanism remaining stable across varying levels of word scrambling. This
  distinction between LLMs' fixed attention patterns primarily focused on word
  form and human readers' adaptive strategy in balancing word form and contextual
  information provides insights into enhancing LLM performance by incorporating
  human-like, context-aware mechanisms.
paper_authors._id: 67c6d22d983375492193aab0
paper_authors.name: Chenxi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T11:16:44.551Z
paper_authors._id: 679bc0ec7f3c28bf968321c8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/679bc0ec7f3c28bf968321c8/uFqCcPbVjDISww46Jbovf.jpeg
paper_authors.fullname: Chenxi Wang
paper_authors.type: user
paper_authors.user: Aurora-cx
paper_authors._id: 67c6d22d983375492193aab1
paper_authors.name: Tianle Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T10:54:29.015Z
paper_authors._id: 6346361c5efccdc07f179cae
paper_authors.avatarUrl: /avatars/217818114a4c19ea4f3e5cdafefb625e.svg
paper_authors.fullname: Gu Tianle
paper_authors.type: user
paper_authors.user: Carol0110
paper_authors._id: 67c6d22d983375492193aab2
paper_authors.name: Zhongyu Wei
paper_authors._id: 67c6d22d983375492193aab3
paper_authors.name: Lang Gao
paper_authors._id: 67c6d22d983375492193aab4
paper_authors.name: Zirui Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T10:17:25.935Z
paper_authors._id: 65407ba7a38390065750233f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65407ba7a38390065750233f/1_IPMZbk-S9u2t18PQgMp.jpeg
paper_authors.fullname: Zirui Song
paper_authors.type: user
paper_authors.user: Ziruibest
paper_authors._id: 67c6d22d983375492193aab5
paper_authors.name: Xiuying Chen
paper_title: Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
paper_summary: Human readers can efficiently comprehend scrambled words, a phenomenon known
  as Typoglycemia, primarily by relying on word form; if word form alone is
  insufficient, they further utilize contextual cues for interpretation. While
  advanced large language models (LLMs) exhibit similar abilities, the underlying
  mechanisms remain unclear. To investigate this, we conduct controlled
  experiments to analyze the roles of word form and contextual information in
  semantic reconstruction and examine LLM attention patterns. Specifically, we
  first propose SemRecScore, a reliable metric to quantify the degree of semantic
  reconstruction, and validate its effectiveness. Using this metric, we study how
  word form and contextual information influence LLMs' semantic reconstruction
  ability, identifying word form as the core factor in this process. Furthermore,
  we analyze how LLMs utilize word form and find that they rely on specialized
  attention heads to extract and process word form information, with this
  mechanism remaining stable across varying levels of word scrambling. This
  distinction between LLMs' fixed attention patterns primarily focused on word
  form and human readers' adaptive strategy in balancing word form and contextual
  information provides insights into enhancing LLM performance by incorporating
  human-like, context-aware mechanisms.