title: BigTrans: Augmenting Large Language Models with Multilingual Translation
    Capability over 100 Languages
summary: Large language models (LLMs) demonstrate promising translation performance
  among various natural languages. However, many LLMs especially the open-sourced
  ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of
  natural languages, making the potential of LLMs on language translation less
  explored. In this work, we present BigTrans which adapts LLaMA that covers only
  20 languages and enhances it with multilingual translation capability on more
  than 100 languages. BigTrans is built upon LLaMA-13B and it is optimized in
  three steps. First, we continue training LLaMA with massive Chinese monolingual
  data. Second, we continue training the model with a large-scale parallel
  dataset that covers 102 natural languages. Third, we instruct-tune the
  foundation model with multilingual translation instructions, leading to our
  BigTrans model. The preliminary experiments on multilingual translation show
  that BigTrans performs comparably with ChatGPT and Google Translate in many
  languages and even outperforms ChatGPT in 8 language pairs. We release the
  BigTrans model and hope it can advance the research progress.
paper_authors._id: 64756221b68461d5cf7ec0e5
paper_authors.name: Wen Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T14:15:21.074Z
paper_authors._id: 641fd72a73cfc036ddbf69c8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641fd72a73cfc036ddbf69c8/W4HG5HRm-OkzVpOotgu3m.jpeg
paper_authors.fullname: WenYang
paper_authors.type: user
paper_authors.user: James-WYang
paper_authors._id: 64756221b68461d5cf7ec0e6
paper_authors.name: Chong Li
paper_authors._id: 64756221b68461d5cf7ec0e7
paper_authors.name: Jiajun Zhang
paper_authors._id: 64756221b68461d5cf7ec0e8
paper_authors.name: Chengqing Zong
paper_title: BigTrans: Augmenting Large Language Models with Multilingual Translation
    Capability over 100 Languages
paper_summary: Large language models (LLMs) demonstrate promising translation performance
  among various natural languages. However, many LLMs especially the open-sourced
  ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of
  natural languages, making the potential of LLMs on language translation less
  explored. In this work, we present BigTrans which adapts LLaMA that covers only
  20 languages and enhances it with multilingual translation capability on more
  than 100 languages. BigTrans is built upon LLaMA-13B and it is optimized in
  three steps. First, we continue training LLaMA with massive Chinese monolingual
  data. Second, we continue training the model with a large-scale parallel
  dataset that covers 102 natural languages. Third, we instruct-tune the
  foundation model with multilingual translation instructions, leading to our
  BigTrans model. The preliminary experiments on multilingual translation show
  that BigTrans performs comparably with ChatGPT and Google Translate in many
  languages and even outperforms ChatGPT in 8 language pairs. We release the
  BigTrans model and hope it can advance the research progress.
paper_ai_keywords: BigTrans LLaMA multilingual translation Chinese monolingual data large-scale parallel dataset instruct-tune ChatGPT Google Translate