title: How Programming Concepts and Neurons Are Shared in Code Language Models
summary: Several studies have explored the mechanisms of large language models (LLMs)
  in coding tasks, but most have focused on programming languages (PLs) in a
  monolingual setting. In this paper, we investigate the relationship between
  multiple PLs and English in the concept space of LLMs. We perform a few-shot
  translation task on 21 PL pairs using two Llama-based models. By decoding the
  embeddings of intermediate layers during this task, we observe that the concept
  space is closer to English (including PL keywords) and assigns high
  probabilities to English tokens in the second half of the intermediate layers.
  We analyze neuron activations for 11 PLs and English, finding that while
  language-specific neurons are primarily concentrated in the bottom layers,
  those exclusive to each PL tend to appear in the top layers. For PLs that are
  highly aligned with multiple other PLs, identifying language-specific neurons
  is not feasible. These PLs also tend to have a larger keyword set than other
  PLs and are closer to the model's concept space regardless of the input/output
  PL in the translation task. Our findings provide insights into how LLMs
  internally represent PLs, revealing structural patterns in the model's concept
  space. Code is available at https://github.com/cisnlp/code-specific-neurons.
paper_authors._id: 683e923450c43488f7ad5e02
paper_authors.name: Amir Hossein Kargaran
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:39:12.648Z
paper_authors._id: 61bf84c8ca59d6d196a1b4e8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61bf84c8ca59d6d196a1b4e8/L_NvUwlMYcye9X35z6f7e.jpeg
paper_authors.fullname: Amir Hossein Kargaran
paper_authors.type: user
paper_authors.user: kargaranamir
paper_authors._id: 683e923450c43488f7ad5e03
paper_authors.name: Yihong Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:28:34.979Z
paper_authors._id: 653f7e569e84d1e8b6a66e70
paper_authors.avatarUrl: /avatars/24eaa6434508a162c349aebfc51990ff.svg
paper_authors.fullname: Yihong Liu
paper_authors.type: user
paper_authors.user: yihongLiu
paper_authors._id: 683e923450c43488f7ad5e04
paper_authors.name: François Yvon
paper_authors._id: 683e923450c43488f7ad5e05
paper_authors.name: Hinrich Schütze
paper_title: How Programming Concepts and Neurons Are Shared in Code Language Models
paper_summary: Several studies have explored the mechanisms of large language models (LLMs)
  in coding tasks, but most have focused on programming languages (PLs) in a
  monolingual setting. In this paper, we investigate the relationship between
  multiple PLs and English in the concept space of LLMs. We perform a few-shot
  translation task on 21 PL pairs using two Llama-based models. By decoding the
  embeddings of intermediate layers during this task, we observe that the concept
  space is closer to English (including PL keywords) and assigns high
  probabilities to English tokens in the second half of the intermediate layers.
  We analyze neuron activations for 11 PLs and English, finding that while
  language-specific neurons are primarily concentrated in the bottom layers,
  those exclusive to each PL tend to appear in the top layers. For PLs that are
  highly aligned with multiple other PLs, identifying language-specific neurons
  is not feasible. These PLs also tend to have a larger keyword set than other
  PLs and are closer to the model's concept space regardless of the input/output
  PL in the translation task. Our findings provide insights into how LLMs
  internally represent PLs, revealing structural patterns in the model's concept
  space. Code is available at https://github.com/cisnlp/code-specific-neurons.
paper_ai_summary: LLMs representing multiple programming languages in their concept space tend to cluster closer to English and exhibit distinct neuron activations for specific languages, particularly in the upper layers, with highly aligned languages sharing similar representations.
paper_ai_keywords: large language models LLMs programming languages PLs concept space few-shot translation embeddings intermediate layers neuron activations language-specific neurons