Language Surgery in Multilingual Large Language Models
Large Language Models (LLMs) have demonstrated remarkable generalization
capabilities across tasks and languages, revolutionizing natural language
processing. This paper investigates the naturally emerging representation
alignment in LLMs, particularly in the middle layers, and its implications for
disentangling language-specific and language-agnostic information. We
empirically confirm the existence of this alignment, analyze its behavior in
comparison to explicitly designed alignment models, and demonstrate its
potential for language-specific manipulation without semantic degradation.
Building on these findings, we propose Inference-Time Language Control (ITLC),
a novel method that leverages latent injection to enable precise cross-lingual
language control and mitigate language confusion in LLMs. Our experiments
highlight ITLC's strong cross-lingual control capabilities while preserving
semantic integrity in target languages. Furthermore, we demonstrate its
effectiveness in alleviating the cross-lingual language confusion problem,
which persists even in current large-scale LLMs, leading to inconsistent
language generation. This work advances our understanding of representation
alignment in LLMs and introduces a practical solution for enhancing their
cross-lingual performance.
6850dfc85e07650ecce890e7
Joanito Agili Lopo
claimed_verified
2025-06-17T07:21:08.064Z
61728a033edf4cc38a81237a
https://cdn-avatars.huggingface.co/v1/production/uploads/1652231681579-61728a033edf4cc38a81237a.jpeg
Joanito Agili Lopo
user
joanitolopo
6850dfc85e07650ecce890e8
Muhammad Ravi Shulthan Habibi
claimed_verified
2025-06-17T07:21:10.071Z
63ddfced5ea8577c8d5fb421
https://cdn-avatars.huggingface.co/v1/production/uploads/1677144169806-63ddfced5ea8577c8d5fb421.jpeg
Muhammad Ravi Shulthan Habibi
user
muhammadravi251001
6850dfc85e07650ecce890e9
Tack Hwa Wong
claimed_verified
2025-06-17T07:21:05.829Z
65a378339b0ac6aafca9bb9c
/avatars/d5e8c2714f025adfe1487384664ddff6.svg
wong tack hwa
user
tackhwa
6850dfc85e07650ecce890ea
Muhammad Ilham Ghozali
6850dfc85e07650ecce890eb
Fajri Koto
6850dfc85e07650ecce890ec
Genta Indra Winata
claimed_verified
2025-06-20T12:20:12.305Z
5f5c4b20e56d546cd6233098
https://cdn-avatars.huggingface.co/v1/production/uploads/1637813888895-5f5c4b20e56d546cd6233098.jpeg
Genta Indra Winata
user
gentaiscool
6850dfc85e07650ecce890ed
Peerat Limkonchotiwat
6850dfc85e07650ecce890ee
Alham Fikri Aji
6850dfc85e07650ecce890ef
Samuel Cahyawijaya
extracted_pending
2025-06-17T03:23:52.870Z
66f1af390ae00cd951861005
/avatars/eeab3bf515e911c3250f99a1a73d43d3.svg
Samuel Cahyawijaya
user
samuel-cahyawijaya
Language Surgery in Multilingual Large Language Models
Large Language Models (LLMs) have demonstrated remarkable generalization
capabilities across tasks and languages, revolutionizing natural language
processing. This paper investigates the naturally emerging representation
alignment in LLMs, particularly in the middle layers, and its implications for
disentangling language-specific and language-agnostic information. We
empirically confirm the existence of this alignment, analyze its behavior in
comparison to explicitly designed alignment models, and demonstrate its
potential for language-specific manipulation without semantic degradation.
Building on these findings, we propose Inference-Time Language Control (ITLC),
a novel method that leverages latent injection to enable precise cross-lingual
language control and mitigate language confusion in LLMs. Our experiments
highlight ITLC's strong cross-lingual control capabilities while preserving
semantic integrity in target languages. Furthermore, we demonstrate its
effectiveness in alleviating the cross-lingual language confusion problem,
which persists even in current large-scale LLMs, leading to inconsistent
language generation. This work advances our understanding of representation
alignment in LLMs and introduces a practical solution for enhancing their
cross-lingual performance.
Research confirms natural representation alignment in large language models and introduces Inference-Time Language Control to enhance cross-lingual performance.
Large Language Models representation alignment middle layers language-specific information language-agnostic information explicitly designed alignment models latent injection cross-lingual language control semantic integrity language confusion