Tuning LLMs with Contrastive Alignment Instructions for Machine
  Translation in Unseen, Low-resource Languages
This article introduces contrastive alignment instructions (AlignInstruct) to
address two challenges in machine translation (MT) on large language models
(LLMs). One is the expansion of supported languages to previously unseen ones.
The second relates to the lack of data in low-resource languages. Model
fine-tuning through MT instructions (MTInstruct) is a straightforward approach
to the first challenge. However, MTInstruct is limited by weak cross-lingual
signals inherent in the second challenge. AlignInstruct emphasizes
cross-lingual supervision via a cross-lingual discriminator built using
statistical word alignments. Our results based on fine-tuning the BLOOMZ models
(1b1, 3b, and 7b1) in up to 24 unseen languages showed that: (1) LLMs can
effectively translate unseen languages using MTInstruct; (2) AlignInstruct led
to consistent improvements in translation quality across 48 translation
directions involving English; (3) Discriminator-based instructions outperformed
their generative counterparts as cross-lingual instructions; (4) AlignInstruct
improved performance in 30 zero-shot directions.
65a0a5e5b1f0788359aa31b7
Zhuoyuan Mao
admin_assigned
2024-01-12T10:43:00.631Z
64365f641c4caf9cc9e1a95a
/avatars/7be333e4f80e668f951d1e4c7351b89b.svg
Zhuoyuan Mao
user
kevinmzy
65a0a5e5b1f0788359aa31b8
Yen Yu
extracted_pending
2024-01-12T02:37:26.368Z
64365f641c4caf9cc9e1a95a
/avatars/7be333e4f80e668f951d1e4c7351b89b.svg
Zhuoyuan Mao
user
kevinmzy
Tuning LLMs with Contrastive Alignment Instructions for Machine
  Translation in Unseen, Low-resource Languages
This article introduces contrastive alignment instructions (AlignInstruct) to
address two challenges in machine translation (MT) on large language models
(LLMs). One is the expansion of supported languages to previously unseen ones.
The second relates to the lack of data in low-resource languages. Model
fine-tuning through MT instructions (MTInstruct) is a straightforward approach
to the first challenge. However, MTInstruct is limited by weak cross-lingual
signals inherent in the second challenge. AlignInstruct emphasizes
cross-lingual supervision via a cross-lingual discriminator built using
statistical word alignments. Our results based on fine-tuning the BLOOMZ models
(1b1, 3b, and 7b1) in up to 24 unseen languages showed that: (1) LLMs can
effectively translate unseen languages using MTInstruct; (2) AlignInstruct led
to consistent improvements in translation quality across 48 translation
directions involving English; (3) Discriminator-based instructions outperformed
their generative counterparts as cross-lingual instructions; (4) AlignInstruct
improved performance in 30 zero-shot directions.
AlignInstruct enhances machine translation quality by introducing cross-lingual supervision using statistical word alignments, outperforming generative methods and improving zero-shot translation capabilities.
contrastive alignment instructions AlignInstruct machine translation large language models LLMs model fine-tuning MT instructions MTInstruct cross-lingual supervision cross-lingual discriminator statistical word alignments BLOOMZ models translation quality zero-shot translation