title: Tuning LLMs with Contrastive Alignment Instructions for Machine
    Translation in Unseen, Low-resource Languages
summary: This article introduces contrastive alignment instructions (AlignInstruct) to
  address two challenges in machine translation (MT) on large language models
  (LLMs). One is the expansion of supported languages to previously unseen ones.
  The second relates to the lack of data in low-resource languages. Model
  fine-tuning through MT instructions (MTInstruct) is a straightforward approach
  to the first challenge. However, MTInstruct is limited by weak cross-lingual
  signals inherent in the second challenge. AlignInstruct emphasizes
  cross-lingual supervision via a cross-lingual discriminator built using
  statistical word alignments. Our results based on fine-tuning the BLOOMZ models
  (1b1, 3b, and 7b1) in up to 24 unseen languages showed that: (1) LLMs can
  effectively translate unseen languages using MTInstruct; (2) AlignInstruct led
  to consistent improvements in translation quality across 48 translation
  directions involving English; (3) Discriminator-based instructions outperformed
  their generative counterparts as cross-lingual instructions; (4) AlignInstruct
  improved performance in 30 zero-shot directions.
paper_authors._id: 65a0a5e5b1f0788359aa31b7
paper_authors.name: Zhuoyuan Mao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-12T10:43:00.631Z
paper_authors._id: 64365f641c4caf9cc9e1a95a
paper_authors.avatarUrl: /avatars/7be333e4f80e668f951d1e4c7351b89b.svg
paper_authors.fullname: Zhuoyuan Mao
paper_authors.type: user
paper_authors.user: kevinmzy
paper_authors._id: 65a0a5e5b1f0788359aa31b8
paper_authors.name: Yen Yu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-01-12T02:37:26.368Z
paper_authors._id: 64365f641c4caf9cc9e1a95a
paper_authors.avatarUrl: /avatars/7be333e4f80e668f951d1e4c7351b89b.svg
paper_authors.fullname: Zhuoyuan Mao
paper_authors.type: user
paper_authors.user: kevinmzy
paper_title: Tuning LLMs with Contrastive Alignment Instructions for Machine
    Translation in Unseen, Low-resource Languages
paper_summary: This article introduces contrastive alignment instructions (AlignInstruct) to
  address two challenges in machine translation (MT) on large language models
  (LLMs). One is the expansion of supported languages to previously unseen ones.
  The second relates to the lack of data in low-resource languages. Model
  fine-tuning through MT instructions (MTInstruct) is a straightforward approach
  to the first challenge. However, MTInstruct is limited by weak cross-lingual
  signals inherent in the second challenge. AlignInstruct emphasizes
  cross-lingual supervision via a cross-lingual discriminator built using
  statistical word alignments. Our results based on fine-tuning the BLOOMZ models
  (1b1, 3b, and 7b1) in up to 24 unseen languages showed that: (1) LLMs can
  effectively translate unseen languages using MTInstruct; (2) AlignInstruct led
  to consistent improvements in translation quality across 48 translation
  directions involving English; (3) Discriminator-based instructions outperformed
  their generative counterparts as cross-lingual instructions; (4) AlignInstruct
  improved performance in 30 zero-shot directions.
paper_ai_summary: AlignInstruct enhances machine translation quality by introducing cross-lingual supervision using statistical word alignments, outperforming generative methods and improving zero-shot translation capabilities.
paper_ai_keywords: contrastive alignment instructions AlignInstruct machine translation large language models LLMs model fine-tuning MT instructions MTInstruct cross-lingual supervision cross-lingual discriminator statistical word alignments BLOOMZ models translation quality zero-shot translation