Contrastive Preference Optimization: Pushing the Boundaries of LLM
  Performance in Machine Translation
Moderate-sized large language models (LLMs) -- those with 7B or 13B
parameters -- exhibit promising machine translation (MT) performance. However,
even the top-performing 13B LLM-based translation models, like ALMA, does not
match the performance of state-of-the-art conventional encoder-decoder
translation models or larger-scale LLMs such as GPT-4. In this study, we bridge
this performance gap. We first assess the shortcomings of supervised
fine-tuning for LLMs in the MT task, emphasizing the quality issues present in
the reference data, despite being human-generated. Then, in contrast to SFT
which mimics reference translations, we introduce Contrastive Preference
Optimization (CPO), a novel approach that trains models to avoid generating
adequate but not perfect translations. Applying CPO to ALMA models with only
22K parallel sentences and 12M parameters yields significant improvements. The
resulting model, called ALMA-R, can match or exceed the performance of the WMT
competition winners and GPT-4 on WMT'21, WMT'22 and WMT'23 test datasets.
65a776634908b2676c709a60
Haoran Xu
admin_assigned
2024-01-17T09:39:13.948Z
61384b860317b0a5c10877d3
https://cdn-avatars.huggingface.co/v1/production/uploads/1631080954171-61384b860317b0a5c10877d3.jpeg
Haoran Xu
user
haoranxu
65a776634908b2676c709a61
Amr Sharaf
admin_assigned
2024-01-17T09:39:21.852Z
619ffce1ea7f7717b42344c6
/avatars/95bbb992cadfc8c3523ec2850d6e1e65.svg
Amr Sharaf
user
amrsharaf
65a776634908b2676c709a62
Yunmo Chen
admin_assigned
2024-01-17T09:40:07.362Z
649ca03443a8bc16c4f39d22
/avatars/2397c073148cd2a1f110858c23837d45.svg
Yunmo Chen
user
yunmochen
65a776634908b2676c709a63
Weiting Tan
65a776634908b2676c709a64
Lingfeng Shen
65a776634908b2676c709a65
Benjamin Van Durme
65a776634908b2676c709a66
Kenton Murray
extracted_pending
2024-01-17T06:40:35.869Z
6137cd80f8e9dca6e152bcd0
/avatars/e2778c566b77d21a849e9dcef9bc686f.svg
Kenton Murray
user
Kenton
65a776634908b2676c709a67
Young Jin Kim
claimed_verified
2024-01-17T08:48:56.566Z
63f5173bb51da4d61da6c038
/avatars/0ee530cf80476aa3985c4d591cd384a1.svg
Young Jin Kim
user
ykim362
Contrastive Preference Optimization: Pushing the Boundaries of LLM
  Performance in Machine Translation
Moderate-sized large language models (LLMs) -- those with 7B or 13B
parameters -- exhibit promising machine translation (MT) performance. However,
even the top-performing 13B LLM-based translation models, like ALMA, does not
match the performance of state-of-the-art conventional encoder-decoder
translation models or larger-scale LLMs such as GPT-4. In this study, we bridge
this performance gap. We first assess the shortcomings of supervised
fine-tuning for LLMs in the MT task, emphasizing the quality issues present in
the reference data, despite being human-generated. Then, in contrast to SFT
which mimics reference translations, we introduce Contrastive Preference
Optimization (CPO), a novel approach that trains models to avoid generating
adequate but not perfect translations. Applying CPO to ALMA models with only
22K parallel sentences and 12M parameters yields significant improvements. The
resulting model, called ALMA-R, can match or exceed the performance of the WMT
competition winners and GPT-4 on WMT'21, WMT'22 and WMT'23 test datasets.
Contrastive Preference Optimization (CPO) enhances moderate-sized LLMs in machine translation, enabling them to match or exceed state-of-the-art model performance.
large language models machine translation supervised fine-tuning contrastive preference optimization ALMA WMT