title: Linear Correlation in LM's Compositional Generalization and
    Hallucination
summary: The generalization of language models (LMs) is undergoing active debates,
  contrasting their potential for general intelligence with their struggles with
  basic knowledge composition (e.g., reverse/transition curse). This paper
  uncovers the phenomenon of linear correlations in LMs during knowledge
  composition. For explanation, there exists a linear transformation between
  certain related knowledge that maps the next token prediction logits from one
  prompt to another, e.g., "X lives in the city of" rightarrow "X lives in the
  country of" for every given X. This mirrors the linearity in human knowledge
  composition, such as Paris rightarrow France. Our findings indicate that the
  linear transformation is resilient to large-scale fine-tuning, generalizing
  updated knowledge when aligned with real-world relationships, but causing
  hallucinations when it deviates. Empirical results suggest that linear
  correlation can serve as a potential identifier of LM's generalization.
  Finally, we show such linear correlations can be learned with a single
  feedforward network and pre-trained vocabulary representations, indicating LM
  generalization heavily relies on the latter.
paper_authors._id: 67a97eea96d822bc6e13a1bb
paper_authors.name: Letian Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:17:52.581Z
paper_authors._id: 64323dd503d81fa4d26deaf9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64323dd503d81fa4d26deaf9/x3ES8VXEZJljxDWvFWaAf.png
paper_authors.fullname: Letian Peng
paper_authors.type: user
paper_authors.user: KomeijiForce
paper_authors._id: 67a97eea96d822bc6e13a1bc
paper_authors.name: Chenyang An
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:18:00.824Z
paper_authors._id: 6546d644bab28a482e1956c3
paper_authors.avatarUrl: /avatars/b35e53afd1acf56534338b7788b49ee1.svg
paper_authors.fullname: Chenyang An
paper_authors.type: user
paper_authors.user: chenyang-an
paper_authors._id: 67a97eea96d822bc6e13a1bd
paper_authors.name: Shibo Hao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:18:07.047Z
paper_authors._id: 660ee5df35d092e3fc2a3685
paper_authors.avatarUrl: /avatars/a7e0472fb7ea49973f74e3eea13dc964.svg
paper_authors.fullname: Shibo Hao
paper_authors.type: user
paper_authors.user: Shibo-UCSD
paper_authors._id: 67a97eea96d822bc6e13a1be
paper_authors.name: Chengyu Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:18:12.963Z
paper_authors._id: 668640a1369b09d564b75509
paper_authors.avatarUrl: /avatars/ef70bfdaae307a602f0ce0a0753596c7.svg
paper_authors.fullname: CHENGYU_DONG
paper_authors.type: user
paper_authors.user: sakuraCY
paper_authors._id: 67a97eea96d822bc6e13a1bf
paper_authors.name: Jingbo Shang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T09:49:54.200Z
paper_authors._id: 660655119e3555d648f6c6b5
paper_authors.avatarUrl: /avatars/ae1e2c97a08be39b77a9f1a5c2a718ef.svg
paper_authors.fullname: Jingbo Shang
paper_authors.type: user
paper_authors.user: shangjingbo
paper_title: Linear Correlation in LM's Compositional Generalization and
    Hallucination
paper_summary: The generalization of language models (LMs) is undergoing active debates,
  contrasting their potential for general intelligence with their struggles with
  basic knowledge composition (e.g., reverse/transition curse). This paper
  uncovers the phenomenon of linear correlations in LMs during knowledge
  composition. For explanation, there exists a linear transformation between
  certain related knowledge that maps the next token prediction logits from one
  prompt to another, e.g., "X lives in the city of" rightarrow "X lives in the
  country of" for every given X. This mirrors the linearity in human knowledge
  composition, such as Paris rightarrow France. Our findings indicate that the
  linear transformation is resilient to large-scale fine-tuning, generalizing
  updated knowledge when aligned with real-world relationships, but causing
  hallucinations when it deviates. Empirical results suggest that linear
  correlation can serve as a potential identifier of LM's generalization.
  Finally, we show such linear correlations can be learned with a single
  feedforward network and pre-trained vocabulary representations, indicating LM
  generalization heavily relies on the latter.
paper_ai_summary: Language models exhibit linear correlations in knowledge composition, resilient to fine-tuning and aligned with real-world relationships, which can be identified and learned using feedforward networks and pre-trained vocabulary.
paper_ai_keywords: linear correlations language models knowledge composition linear transformation feedforward network pre-trained vocabulary