title: Marco-LLM: Bridging Languages via Massive Multilingual Training for
    Cross-Lingual Enhancement
summary: Large Language Models (LLMs) have achieved remarkable progress in recent
  years; however, their excellent performance is still largely limited to major
  world languages, primarily English. Many LLMs continue to face challenges with
  multilingual tasks, especially when it comes to low-resource languages. To
  address this issue, we introduced Marco-LLM: Massive multilingual training for
  cross-lingual enhancement LLM. We have collected a substantial amount of
  multilingual data for several low-resource languages and conducted extensive
  continual pre-training using the Qwen2 models. This effort has resulted in a
  multilingual LLM named Marco-LLM. Through comprehensive evaluations on various
  multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA
  and many others, Marco-LLM has demonstrated substantial improvements over
  state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements
  in any-to-any machine translation tasks, showing the effectiveness of our
  multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not
  only perform exceptionally well in multilingual tasks, including low-resource
  languages, but also maintain strong performance in English and other major
  languages, closing the performance gap between high- and low-resource language
  capabilities. By bridging languages, this effort demonstrates our dedication to
  ensuring LLMs work accurately across various languages.
paper_authors._id: 6752a3acd34997a5fbc79614
paper_authors.name: Lingfeng Ming
paper_authors._id: 6752a3acd34997a5fbc79615
paper_authors.name: Bo Zeng
paper_authors._id: 6752a3acd34997a5fbc79616
paper_authors.name: Chenyang Lyu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:39:28.388Z
paper_authors._id: 6527d8b077bceabaab382a75
paper_authors.avatarUrl: /avatars/69caacf9153dbf6a3796693a968b363f.svg
paper_authors.fullname: Chenyang Lyu
paper_authors.type: user
paper_authors.user: ChenyangLyu
paper_authors._id: 6752a3acd34997a5fbc79617
paper_authors.name: Tianqi Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-28T12:17:01.903Z
paper_authors._id: 66389e004ea0d8fe73853fea
paper_authors.avatarUrl: /avatars/1b9bd9e64b9f3e2660f597ab65f3c15a.svg
paper_authors.fullname: ShiTianqi
paper_authors.type: user
paper_authors.user: shitianqi
paper_authors._id: 6752a3acd34997a5fbc79618
paper_authors.name: Yu Zhao
paper_authors._id: 6752a3acd34997a5fbc79619
paper_authors.name: Xue Yang
paper_authors._id: 6752a3acd34997a5fbc7961a
paper_authors.name: Yefeng Liu
paper_authors._id: 6752a3acd34997a5fbc7961b
paper_authors.name: Yiyu Wang
paper_authors._id: 6752a3acd34997a5fbc7961c
paper_authors.name: Linlong Xu
paper_authors._id: 6752a3acd34997a5fbc7961d
paper_authors.name: Yangyang Liu
paper_authors._id: 6752a3acd34997a5fbc7961e
paper_authors.name: Xiaohu Zhao
paper_authors._id: 6752a3acd34997a5fbc7961f
paper_authors.name: Hao Wang
paper_authors._id: 6752a3acd34997a5fbc79620
paper_authors.name: Heng Liu
paper_authors._id: 6752a3acd34997a5fbc79621
paper_authors.name: Hao Zhou
paper_authors._id: 6752a3acd34997a5fbc79622
paper_authors.name: Huifeng Yin
paper_authors._id: 6752a3acd34997a5fbc79623
paper_authors.name: Zifu Shang
paper_authors._id: 6752a3acd34997a5fbc79624
paper_authors.name: Haijun Li
paper_authors._id: 6752a3acd34997a5fbc79625
paper_authors.name: Longyue Wang
paper_authors._id: 6752a3acd34997a5fbc79626
paper_authors.name: Weihua Luo
paper_authors._id: 6752a3acd34997a5fbc79627
paper_authors.name: Kaifu Zhang
paper_title: Marco-LLM: Bridging Languages via Massive Multilingual Training for
    Cross-Lingual Enhancement
paper_summary: Large Language Models (LLMs) have achieved remarkable progress in recent
  years; however, their excellent performance is still largely limited to major
  world languages, primarily English. Many LLMs continue to face challenges with
  multilingual tasks, especially when it comes to low-resource languages. To
  address this issue, we introduced Marco-LLM: Massive multilingual training for
  cross-lingual enhancement LLM. We have collected a substantial amount of
  multilingual data for several low-resource languages and conducted extensive
  continual pre-training using the Qwen2 models. This effort has resulted in a
  multilingual LLM named Marco-LLM. Through comprehensive evaluations on various
  multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA
  and many others, Marco-LLM has demonstrated substantial improvements over
  state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements
  in any-to-any machine translation tasks, showing the effectiveness of our
  multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not
  only perform exceptionally well in multilingual tasks, including low-resource
  languages, but also maintain strong performance in English and other major
  languages, closing the performance gap between high- and low-resource language
  capabilities. By bridging languages, this effort demonstrates our dedication to
  ensuring LLMs work accurately across various languages.
paper_ai_summary: Marco-LLM, a multilingual LLM trained on diverse datasets, significantly improves performance across various low-resource languages and maintains strong capabilities in major languages.
paper_ai_keywords: LLMs Marco-LLM multilingual training cross-lingual enhancement Qwen2 multilingual benchmarks MMMLU AGIEval Belebele Flores-200 XCOPA machine translation