title: On Relation-Specific Neurons in Large Language Models
summary: In large language models (LLMs), certain neurons can store distinct pieces of
  knowledge learned during pretraining. While knowledge typically appears as a
  combination of relations and entities, it remains unclear whether some neurons
  focus on a relation itself -- independent of any entity. We hypothesize such
  neurons detect a relation in the input text and guide generation involving such
  a relation. To investigate this, we study the Llama-2 family on a chosen set of
  relations with a statistics-based method. Our experiments demonstrate the
  existence of relation-specific neurons. We measure the effect of selectively
  deactivating candidate neurons specific to relation r on the LLM's ability to
  handle (1) facts whose relation is r and (2) facts whose relation is a
  different relation r' neq r. With respect to their capacity for encoding
  relation information, we give evidence for the following three properties of
  relation-specific neurons. (i) Neuron cumulativity. The neurons for
  r present a cumulative effect so that deactivating a larger portion of them
  results in the degradation of more facts in r. (ii) Neuron
  versatility. Neurons can be shared across multiple closely related as well as
  less related relations. Some relation neurons transfer across languages.
  (iii) Neuron interference. Deactivating neurons specific to one
  relation can improve LLM generation performance for facts of other relations.
  We will make our code publicly available at
  https://github.com/cisnlp/relation-specific-neurons.
paper_authors._id: 67bf1808b91e7e6477d92c1e
paper_authors.name: Yihong Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-28T15:14:48.351Z
paper_authors._id: 653f7e569e84d1e8b6a66e70
paper_authors.avatarUrl: /avatars/24eaa6434508a162c349aebfc51990ff.svg
paper_authors.fullname: Yihong Liu
paper_authors.type: user
paper_authors.user: yihongLiu
paper_authors._id: 67bf1808b91e7e6477d92c1f
paper_authors.name: Runsheng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-28T15:16:28.041Z
paper_authors._id: 63629b9f2a84d82a8c8feb32
paper_authors.avatarUrl: /avatars/8484b5bf8311b28249757729b1ce80f8.svg
paper_authors.fullname: Chen
paper_authors.type: user
paper_authors.user: Runsheng
paper_authors._id: 67bf1808b91e7e6477d92c20
paper_authors.name: Lea Hirlimann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-28T15:16:18.398Z
paper_authors._id: 658559148615630cb3ec5b6b
paper_authors.avatarUrl: /avatars/dd804ca277e6b19903bb550cc167ba4a.svg
paper_authors.fullname: Lea Hirlimann
paper_authors.type: user
paper_authors.user: hirlimann
paper_authors._id: 67bf1808b91e7e6477d92c21
paper_authors.name: Ahmad Dawar Hakimi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-28T15:16:11.693Z
paper_authors._id: 62502669d2d191ac43320ade
paper_authors.avatarUrl: /avatars/7997e9b2012059edb22b745c3b737481.svg
paper_authors.fullname: Ahmad Dawar Hakimi
paper_authors.type: user
paper_authors.user: adhakimi
paper_authors._id: 67bf1808b91e7e6477d92c22
paper_authors.name: Mingyang Wang
paper_authors._id: 67bf1808b91e7e6477d92c23
paper_authors.name: Amir Hossein Kargaran
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T15:37:07.932Z
paper_authors._id: 61bf84c8ca59d6d196a1b4e8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61bf84c8ca59d6d196a1b4e8/L_NvUwlMYcye9X35z6f7e.jpeg
paper_authors.fullname: Amir Hossein Kargaran
paper_authors.type: user
paper_authors.user: kargaranamir
paper_authors._id: 67bf1808b91e7e6477d92c24
paper_authors.name: Sascha Rothe
paper_authors._id: 67bf1808b91e7e6477d92c25
paper_authors.name: François Yvon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-28T15:16:57.343Z
paper_authors._id: 62ab10f04bd2ebf5dbad205c
paper_authors.avatarUrl: /avatars/65356b3b057159cc67a86efb26b53486.svg
paper_authors.fullname: François Yvon
paper_authors.type: user
paper_authors.user: fyvo
paper_authors._id: 67bf1808b91e7e6477d92c26
paper_authors.name: Hinrich Schütze
paper_title: On Relation-Specific Neurons in Large Language Models
paper_summary: In large language models (LLMs), certain neurons can store distinct pieces of
  knowledge learned during pretraining. While knowledge typically appears as a
  combination of relations and entities, it remains unclear whether some neurons
  focus on a relation itself -- independent of any entity. We hypothesize such
  neurons detect a relation in the input text and guide generation involving such
  a relation. To investigate this, we study the Llama-2 family on a chosen set of
  relations with a statistics-based method. Our experiments demonstrate the
  existence of relation-specific neurons. We measure the effect of selectively
  deactivating candidate neurons specific to relation r on the LLM's ability to
  handle (1) facts whose relation is r and (2) facts whose relation is a
  different relation r' neq r. With respect to their capacity for encoding
  relation information, we give evidence for the following three properties of
  relation-specific neurons. (i) Neuron cumulativity. The neurons for
  r present a cumulative effect so that deactivating a larger portion of them
  results in the degradation of more facts in r. (ii) Neuron
  versatility. Neurons can be shared across multiple closely related as well as
  less related relations. Some relation neurons transfer across languages.
  (iii) Neuron interference. Deactivating neurons specific to one
  relation can improve LLM generation performance for facts of other relations.
  We will make our code publicly available at
  https://github.com/cisnlp/relation-specific-neurons.
paper_ai_summary: Studies on the Llama-2 family reveal relation-specific neurons that cumulatively encode relation information, are versatile across related relations, and can interfere with each other, impacting LLM performance.
paper_ai_keywords: large language models LLMs relation-specific neurons pretraining knowledge storage relation detection neuron cumulativity neuron versatility neuron interference