title: Contrastive Preference Optimization: Pushing the Boundaries of LLM
    Performance in Machine Translation
summary: Moderate-sized large language models (LLMs) -- those with 7B or 13B
  parameters -- exhibit promising machine translation (MT) performance. However,
  even the top-performing 13B LLM-based translation models, like ALMA, does not
  match the performance of state-of-the-art conventional encoder-decoder
  translation models or larger-scale LLMs such as GPT-4. In this study, we bridge
  this performance gap. We first assess the shortcomings of supervised
  fine-tuning for LLMs in the MT task, emphasizing the quality issues present in
  the reference data, despite being human-generated. Then, in contrast to SFT
  which mimics reference translations, we introduce Contrastive Preference
  Optimization (CPO), a novel approach that trains models to avoid generating
  adequate but not perfect translations. Applying CPO to ALMA models with only
  22K parallel sentences and 12M parameters yields significant improvements. The
  resulting model, called ALMA-R, can match or exceed the performance of the WMT
  competition winners and GPT-4 on WMT'21, WMT'22 and WMT'23 test datasets.
paper_authors._id: 65a776634908b2676c709a60
paper_authors.name: Haoran Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-17T09:39:13.948Z
paper_authors._id: 61384b860317b0a5c10877d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631080954171-61384b860317b0a5c10877d3.jpeg
paper_authors.fullname: Haoran Xu
paper_authors.type: user
paper_authors.user: haoranxu
paper_authors._id: 65a776634908b2676c709a61
paper_authors.name: Amr Sharaf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-17T09:39:21.852Z
paper_authors._id: 619ffce1ea7f7717b42344c6
paper_authors.avatarUrl: /avatars/95bbb992cadfc8c3523ec2850d6e1e65.svg
paper_authors.fullname: Amr Sharaf
paper_authors.type: user
paper_authors.user: amrsharaf
paper_authors._id: 65a776634908b2676c709a62
paper_authors.name: Yunmo Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-17T09:40:07.362Z
paper_authors._id: 649ca03443a8bc16c4f39d22
paper_authors.avatarUrl: /avatars/5c6ac56171a24ee16098bf9c8dcd82f7.svg
paper_authors.fullname: Yunmo Chen
paper_authors.type: user
paper_authors.user: yunmochen
paper_authors._id: 65a776634908b2676c709a63
paper_authors.name: Weiting Tan
paper_authors._id: 65a776634908b2676c709a64
paper_authors.name: Lingfeng Shen
paper_authors._id: 65a776634908b2676c709a65
paper_authors.name: Benjamin Van Durme
paper_authors._id: 65a776634908b2676c709a66
paper_authors.name: Kenton Murray
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-01-17T06:40:35.869Z
paper_authors._id: 6137cd80f8e9dca6e152bcd0
paper_authors.avatarUrl: /avatars/e2778c566b77d21a849e9dcef9bc686f.svg
paper_authors.fullname: Kenton Murray
paper_authors.type: user
paper_authors.user: Kenton
paper_authors._id: 65a776634908b2676c709a67
paper_authors.name: Young Jin Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-17T08:48:56.566Z
paper_authors._id: 63f5173bb51da4d61da6c038
paper_authors.avatarUrl: /avatars/0ee530cf80476aa3985c4d591cd384a1.svg
paper_authors.fullname: Young Jin Kim
paper_authors.type: user
paper_authors.user: ykim362
paper_title: Contrastive Preference Optimization: Pushing the Boundaries of LLM
    Performance in Machine Translation
paper_summary: Moderate-sized large language models (LLMs) -- those with 7B or 13B
  parameters -- exhibit promising machine translation (MT) performance. However,
  even the top-performing 13B LLM-based translation models, like ALMA, does not
  match the performance of state-of-the-art conventional encoder-decoder
  translation models or larger-scale LLMs such as GPT-4. In this study, we bridge
  this performance gap. We first assess the shortcomings of supervised
  fine-tuning for LLMs in the MT task, emphasizing the quality issues present in
  the reference data, despite being human-generated. Then, in contrast to SFT
  which mimics reference translations, we introduce Contrastive Preference
  Optimization (CPO), a novel approach that trains models to avoid generating
  adequate but not perfect translations. Applying CPO to ALMA models with only
  22K parallel sentences and 12M parameters yields significant improvements. The
  resulting model, called ALMA-R, can match or exceed the performance of the WMT
  competition winners and GPT-4 on WMT'21, WMT'22 and WMT'23 test datasets.
paper_ai_summary: Contrastive Preference Optimization (CPO) enhances moderate-sized LLMs in machine translation, enabling them to match or exceed state-of-the-art model performance.
paper_ai_keywords: large language models machine translation supervised fine-tuning contrastive preference optimization ALMA WMT