title: How does Alignment Enhance LLMs' Multilingual Capabilities? A Language
    Neurons Perspective
summary: Multilingual Alignment is an effective and representative paradigm to enhance
  LLMs' multilingual capabilities, which transfers the capabilities from the
  high-resource languages to the low-resource languages. Meanwhile, some
  researches on language-specific neurons reveal that there are language-specific
  neurons that are selectively activated in LLMs when processing different
  languages. This provides a new perspective to analyze and understand LLMs'
  mechanisms more specifically in multilingual scenarios. In this work, we
  propose a new finer-grained neuron identification algorithm, which detects
  language neurons~(including language-specific neurons and language-related
  neurons) and language-agnostic neurons. Furthermore, based on the
  distributional characteristics of different types of neurons, we divide the
  LLMs' internal process for multilingual inference into four parts: (1)
  multilingual understanding, (2) shared semantic space reasoning, (3)
  multilingual output space transformation, and (4) vocabulary space outputting.
  Additionally, we systematically analyze the models before and after alignment
  with a focus on different types of neurons. We also analyze the phenomenon of
  ''Spontaneous Multilingual Alignment''. Overall, our work conducts a
  comprehensive investigation based on different types of neurons, providing
  empirical results and valuable insights for better understanding multilingual
  alignment and multilingual capabilities of LLMs.
paper_authors._id: 68369d9f8a36b9fa7f340c86
paper_authors.name: Shimao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:56:40.548Z
paper_authors._id: 65080dc63fc966d1bbba485d
paper_authors.avatarUrl: /avatars/347890233f2316e7f7a04d652b2378bb.svg
paper_authors.fullname: Shimao Zhang
paper_authors.type: user
paper_authors.user: Shimao-Zhang
paper_authors._id: 68369d9f8a36b9fa7f340c87
paper_authors.name: Zhejian Lai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:56:38.385Z
paper_authors._id: 643525ea0b30bd434ea15363
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643525ea0b30bd434ea15363/7sAzllfWUPtt68NY1gDLj.png
paper_authors.fullname: Jackie Lai
paper_authors.type: user
paper_authors.user: DreamW1ngs
paper_authors._id: 68369d9f8a36b9fa7f340c88
paper_authors.name: Xiang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T12:12:17.068Z
paper_authors._id: 65cc78118ebd392213fc9e16
paper_authors.avatarUrl: /avatars/859735d5abdb6f2c6f26bd8db034a1ce.svg
paper_authors.fullname: Xiang Liu
paper_authors.type: user
paper_authors.user: VincentLx
paper_authors._id: 68369d9f8a36b9fa7f340c89
paper_authors.name: Shuaijie She
paper_authors._id: 68369d9f8a36b9fa7f340c8a
paper_authors.name: Xiao Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T20:18:53.657Z
paper_authors._id: 63fb6e281b4b1bd4e7ffc5be
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677422062937-noauth.jpeg
paper_authors.fullname: Xiao Liu
paper_authors.type: user
paper_authors.user: lx865712528
paper_authors._id: 68369d9f8a36b9fa7f340c8b
paper_authors.name: Yeyun Gong
paper_authors._id: 68369d9f8a36b9fa7f340c8c
paper_authors.name: Shujian Huang
paper_authors._id: 68369d9f8a36b9fa7f340c8d
paper_authors.name: Jiajun Chen
paper_title: How does Alignment Enhance LLMs' Multilingual Capabilities? A Language
    Neurons Perspective
paper_summary: Multilingual Alignment is an effective and representative paradigm to enhance
  LLMs' multilingual capabilities, which transfers the capabilities from the
  high-resource languages to the low-resource languages. Meanwhile, some
  researches on language-specific neurons reveal that there are language-specific
  neurons that are selectively activated in LLMs when processing different
  languages. This provides a new perspective to analyze and understand LLMs'
  mechanisms more specifically in multilingual scenarios. In this work, we
  propose a new finer-grained neuron identification algorithm, which detects
  language neurons~(including language-specific neurons and language-related
  neurons) and language-agnostic neurons. Furthermore, based on the
  distributional characteristics of different types of neurons, we divide the
  LLMs' internal process for multilingual inference into four parts: (1)
  multilingual understanding, (2) shared semantic space reasoning, (3)
  multilingual output space transformation, and (4) vocabulary space outputting.
  Additionally, we systematically analyze the models before and after alignment
  with a focus on different types of neurons. We also analyze the phenomenon of
  ''Spontaneous Multilingual Alignment''. Overall, our work conducts a
  comprehensive investigation based on different types of neurons, providing
  empirical results and valuable insights for better understanding multilingual
  alignment and multilingual capabilities of LLMs.
paper_ai_summary: The research proposes a finer-grained neuron identification algorithm for detecting language-specific and language-agnostic neurons in LLMs, and investigates the impact on multilingual alignment and capabilities through analysis of multilingual understanding, shared semantic reasoning, multilingual output transformation, and vocabulary space outputting.
paper_ai_keywords: multilingual alignment language-specific neurons language-agnostic neurons shared semantic space multilingual output space vocabulary space neuron identification algorithm spontaneous multilingual alignment