title: A Paradigm Shift in Machine Translation: Boosting Translation
    Performance of Large Language Models
summary: Generative Large Language Models (LLMs) have achieved remarkable advancements
  in various NLP tasks. However, these advances have not been reflected in the
  translation task, especially those with moderate model sizes (i.e., 7B or 13B
  parameters), which still lag behind conventional supervised encoder-decoder
  translation models. Previous studies have attempted to improve the translation
  capabilities of these moderate LLMs, but their gains have been limited. In this
  study, we propose a novel fine-tuning approach for LLMs that is specifically
  designed for the translation task, eliminating the need for the abundant
  parallel data that traditional translation models usually depend on. Our
  approach consists of two fine-tuning stages: initial fine-tuning on monolingual
  data followed by subsequent fine-tuning on a small set of high-quality parallel
  data. We introduce the LLM developed through this strategy as Advanced Language
  Model-based trAnslator (ALMA). Based on LLaMA-2 as our underlying model, our
  results show that the model can achieve an average improvement of more than 12
  BLEU and 12 COMET over its zero-shot performance across 10 translation
  directions from the WMT'21 (2 directions) and WMT'22 (8 directions) test
  datasets. The performance is significantly better than all prior work and even
  superior to the NLLB-54B model and GPT-3.5-text-davinci-003, with only 7B or
  13B parameters. This method establishes the foundation for a novel training
  paradigm in machine translation.
paper_authors._id: 650ce69e23e8028a895c6259
paper_authors.name: Haoran Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:15:51.545Z
paper_authors._id: 61384b860317b0a5c10877d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631080954171-61384b860317b0a5c10877d3.jpeg
paper_authors.fullname: Haoran Xu
paper_authors.type: user
paper_authors.user: haoranxu
paper_authors._id: 650ce69e23e8028a895c625a
paper_authors.name: Young Jin Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:16:40.868Z
paper_authors._id: 63f5173bb51da4d61da6c038
paper_authors.avatarUrl: /avatars/0ee530cf80476aa3985c4d591cd384a1.svg
paper_authors.fullname: Young Jin Kim
paper_authors.type: user
paper_authors.user: ykim362
paper_authors._id: 650ce69e23e8028a895c625b
paper_authors.name: Amr Sharaf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:16:49.141Z
paper_authors._id: 619ffce1ea7f7717b42344c6
paper_authors.avatarUrl: /avatars/95bbb992cadfc8c3523ec2850d6e1e65.svg
paper_authors.fullname: Amr Sharaf
paper_authors.type: user
paper_authors.user: amrsharaf
paper_authors._id: 650ce69e23e8028a895c625c
paper_authors.name: Hany Hassan Awadalla
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-22T17:53:43.613Z
paper_authors._id: 6290f299d1be2630c0d30e1c
paper_authors.avatarUrl: /avatars/6aff70159d1930732d5ffeed54761351.svg
paper_authors.fullname: Hany
paper_authors.type: user
paper_authors.user: hanyh
paper_title: A Paradigm Shift in Machine Translation: Boosting Translation
    Performance of Large Language Models
paper_summary: Generative Large Language Models (LLMs) have achieved remarkable advancements
  in various NLP tasks. However, these advances have not been reflected in the
  translation task, especially those with moderate model sizes (i.e., 7B or 13B
  parameters), which still lag behind conventional supervised encoder-decoder
  translation models. Previous studies have attempted to improve the translation
  capabilities of these moderate LLMs, but their gains have been limited. In this
  study, we propose a novel fine-tuning approach for LLMs that is specifically
  designed for the translation task, eliminating the need for the abundant
  parallel data that traditional translation models usually depend on. Our
  approach consists of two fine-tuning stages: initial fine-tuning on monolingual
  data followed by subsequent fine-tuning on a small set of high-quality parallel
  data. We introduce the LLM developed through this strategy as Advanced Language
  Model-based trAnslator (ALMA). Based on LLaMA-2 as our underlying model, our
  results show that the model can achieve an average improvement of more than 12
  BLEU and 12 COMET over its zero-shot performance across 10 translation
  directions from the WMT'21 (2 directions) and WMT'22 (8 directions) test
  datasets. The performance is significantly better than all prior work and even
  superior to the NLLB-54B model and GPT-3.5-text-davinci-003, with only 7B or
  13B parameters. This method establishes the foundation for a novel training
  paradigm in machine translation.
paper_ai_summary: A novel fine-tuning approach for moderate-sized language models significantly improves translation performance with less parallel data compared to traditional methods.
paper_ai_keywords: Generative Large Language Models fine-tuning monolingual data parallel data Advanced Language Model-based trAnslator (ALMA) zero-shot performance BLEU COMET WMT'21 WMT'22 NLLB-54B GPT-3.5-text-davinci-003