Marco-LLM: Bridging Languages via Massive Multilingual Training for
  Cross-Lingual Enhancement
Large Language Models (LLMs) have achieved remarkable progress in recent
years; however, their excellent performance is still largely limited to major
world languages, primarily English. Many LLMs continue to face challenges with
multilingual tasks, especially when it comes to low-resource languages. To
address this issue, we introduced Marco-LLM: Massive multilingual training for
cross-lingual enhancement LLM. We have collected a substantial amount of
multilingual data for several low-resource languages and conducted extensive
continual pre-training using the Qwen2 models. This effort has resulted in a
multilingual LLM named Marco-LLM. Through comprehensive evaluations on various
multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA
and many others, Marco-LLM has demonstrated substantial improvements over
state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements
in any-to-any machine translation tasks, showing the effectiveness of our
multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not
only perform exceptionally well in multilingual tasks, including low-resource
languages, but also maintain strong performance in English and other major
languages, closing the performance gap between high- and low-resource language
capabilities. By bridging languages, this effort demonstrates our dedication to
ensuring LLMs work accurately across various languages.
6752a3acd34997a5fbc79614
Lingfeng Ming
6752a3acd34997a5fbc79615
Bo Zeng
6752a3acd34997a5fbc79616
Chenyang Lyu
claimed_verified
2025-04-23T08:39:28.388Z
6527d8b077bceabaab382a75
/avatars/69caacf9153dbf6a3796693a968b363f.svg
Chenyang Lyu
user
ChenyangLyu
6752a3acd34997a5fbc79617
Tianqi Shi
claimed_verified
2025-02-28T12:17:01.903Z
66389e004ea0d8fe73853fea
/avatars/1b9bd9e64b9f3e2660f597ab65f3c15a.svg
ShiTianqi
user
shitianqi
6752a3acd34997a5fbc79618
Yu Zhao
6752a3acd34997a5fbc79619
Xue Yang
6752a3acd34997a5fbc7961a
Yefeng Liu
6752a3acd34997a5fbc7961b
Yiyu Wang
6752a3acd34997a5fbc7961c
Linlong Xu
6752a3acd34997a5fbc7961d
Yangyang Liu
6752a3acd34997a5fbc7961e
Xiaohu Zhao
6752a3acd34997a5fbc7961f
Hao Wang
6752a3acd34997a5fbc79620
Heng Liu
6752a3acd34997a5fbc79621
Hao Zhou
6752a3acd34997a5fbc79622
Huifeng Yin
6752a3acd34997a5fbc79623
Zifu Shang
6752a3acd34997a5fbc79624
Haijun Li
6752a3acd34997a5fbc79625
Longyue Wang
6752a3acd34997a5fbc79626
Weihua Luo
6752a3acd34997a5fbc79627
Kaifu Zhang
Marco-LLM: Bridging Languages via Massive Multilingual Training for
  Cross-Lingual Enhancement
Large Language Models (LLMs) have achieved remarkable progress in recent
years; however, their excellent performance is still largely limited to major
world languages, primarily English. Many LLMs continue to face challenges with
multilingual tasks, especially when it comes to low-resource languages. To
address this issue, we introduced Marco-LLM: Massive multilingual training for
cross-lingual enhancement LLM. We have collected a substantial amount of
multilingual data for several low-resource languages and conducted extensive
continual pre-training using the Qwen2 models. This effort has resulted in a
multilingual LLM named Marco-LLM. Through comprehensive evaluations on various
multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA
and many others, Marco-LLM has demonstrated substantial improvements over
state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements
in any-to-any machine translation tasks, showing the effectiveness of our
multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not
only perform exceptionally well in multilingual tasks, including low-resource
languages, but also maintain strong performance in English and other major
languages, closing the performance gap between high- and low-resource language
capabilities. By bridging languages, this effort demonstrates our dedication to
ensuring LLMs work accurately across various languages.
Marco-LLM, a multilingual LLM trained on diverse datasets, significantly improves performance across various low-resource languages and maintains strong capabilities in major languages.
LLMs Marco-LLM multilingual training cross-lingual enhancement Qwen2 multilingual benchmarks MMMLU AGIEval Belebele Flores-200 XCOPA machine translation