Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysis
<|reference_start|>Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysis: Probing techniques for large language models (LLMs) have primarily focused on English, overlooking the vast majority of the world's languages. In this paper, we extend these probing methods to a multilingual context, investigating the behaviors of LLMs across diverse languages. We conduct experiments on several open-source LLM models, analyzing probing accuracy, trends across layers, and similarities between probing vectors for multiple languages. Our key findings reveal: (1) a consistent performance gap between high-resource and low-resource languages, with high-resource languages achieving significantly higher probing accuracy; (2) divergent layer-wise accuracy trends, where high-resource languages show substantial improvement in deeper layers similar to English; and (3) higher representational similarities among high-resource languages, with low-resource languages demonstrating lower similarities both among themselves and with high-resource languages. These results highlight significant disparities in LLMs' multilingual capabilities and emphasize the need for improved modeling of low-resource languages.<|reference_end|>
@article{li2024exploring,
    title={Exploring Multilingual Probing in Large Language Models: A
  Cross-Language Analysis},
    author={Daoyang Li, Mingyu Jin, Qingcheng Zeng, Haiyan Zhao, Mengnan Du},
    journal={arXiv preprint arXiv:2409.14459},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.14459},
    primaryClass={cs.CL cs.AI cs.LG}
}