title: Is Preference Alignment Always the Best Option to Enhance LLM-Based
    Translation? An Empirical Analysis
summary: Neural metrics for machine translation (MT) evaluation have become
  increasingly prominent due to their superior correlation with human judgments
  compared to traditional lexical metrics. Researchers have therefore utilized
  neural metrics through quality-informed decoding strategies, achieving better
  results than likelihood-based methods. With the rise of Large Language Models
  (LLMs), preference-based alignment techniques have gained attention for their
  potential to enhance translation quality by optimizing model weights directly
  on preferences induced by quality estimators. This study focuses on Contrastive
  Preference Optimization (CPO) and conducts extensive experiments to evaluate
  the impact of preference-based alignment on translation quality. Our findings
  indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
  on high-quality data with regard to the alignment metric, it may lead to
  instability across downstream evaluation metrics, particularly between neural
  and lexical ones. Additionally, we demonstrate that relying solely on the base
  model for generating candidate translations achieves performance comparable to
  using multiple external systems, while ensuring better consistency across
  downstream metrics.
paper_authors._id: 66fd33a20cde4879f9ab4654
paper_authors.name: Hippolyte Gisserot-Boukhlef
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-02T12:44:47.000Z
paper_authors._id: 65fa95405355a52c784633fc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65fa95405355a52c784633fc/rSfBUHPa7eSAsLd8DuOq4.png
paper_authors.fullname: Hippolyte Gisserot-Boukhlef
paper_authors.type: user
paper_authors.user: hgissbkh
paper_authors._id: 66fd33a20cde4879f9ab4655
paper_authors.name: Ricardo Rei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:23:48.804Z
paper_authors._id: 60d9a9791fa5d458da77754d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1624887555363-60d9a9791fa5d458da77754d.jpeg
paper_authors.fullname: Ricardo Rei
paper_authors.type: user
paper_authors.user: RicardoRei
paper_authors._id: 66fd33a20cde4879f9ab4656
paper_authors.name: Emmanuel Malherbe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:53:40.295Z
paper_authors._id: 66f2d6a684a241caac8e16dc
paper_authors.avatarUrl: /avatars/81acb87c2b07bea938251b40a2139911.svg
paper_authors.fullname: Emmanuel Malherbe
paper_authors.type: user
paper_authors.user: emmanuelmalherbe
paper_authors._id: 66fd33a20cde4879f9ab4657
paper_authors.name: Céline Hudelot
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:53:47.051Z
paper_authors._id: 61efea03a57920a251ec19b8
paper_authors.avatarUrl: /avatars/f47c8e3cb17a2bf7d43f2c152bb86885.svg
paper_authors.fullname: Celine Hudelot
paper_authors.type: user
paper_authors.user: CelineH
paper_authors._id: 66fd33a20cde4879f9ab4658
paper_authors.name: Pierre Colombo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:53:53.386Z
paper_authors._id: 644a900e3a619fe72b14af0f
paper_authors.avatarUrl: /avatars/e2d5dac3d92757ed48e37e126a3464a3.svg
paper_authors.fullname: Colombo
paper_authors.type: user
paper_authors.user: PierreColombo
paper_authors._id: 66fd33a20cde4879f9ab4659
paper_authors.name: Nuno M. Guerreiro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:54:03.564Z
paper_authors._id: 61f06c8ba57920a251ec1a5c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61f06c8ba57920a251ec1a5c/TmmSN9xWm1noLeAtsUSRk.jpeg
paper_authors.fullname: Nuno Guerreiro
paper_authors.type: user
paper_authors.user: nunonmg
paper_title: Is Preference Alignment Always the Best Option to Enhance LLM-Based
    Translation? An Empirical Analysis
paper_summary: Neural metrics for machine translation (MT) evaluation have become
  increasingly prominent due to their superior correlation with human judgments
  compared to traditional lexical metrics. Researchers have therefore utilized
  neural metrics through quality-informed decoding strategies, achieving better
  results than likelihood-based methods. With the rise of Large Language Models
  (LLMs), preference-based alignment techniques have gained attention for their
  potential to enhance translation quality by optimizing model weights directly
  on preferences induced by quality estimators. This study focuses on Contrastive
  Preference Optimization (CPO) and conducts extensive experiments to evaluate
  the impact of preference-based alignment on translation quality. Our findings
  indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
  on high-quality data with regard to the alignment metric, it may lead to
  instability across downstream evaluation metrics, particularly between neural
  and lexical ones. Additionally, we demonstrate that relying solely on the base
  model for generating candidate translations achieves performance comparable to
  using multiple external systems, while ensuring better consistency across
  downstream metrics.
paper_ai_summary: Contrastive Preference Optimization enhances translation quality on high-quality data but may cause instability across different evaluation metrics, while the base model's performance is comparable to using multiple external systems.
paper_ai_keywords: neural metrics machine translation quality-informed decoding likelihood-based methods Large Language Models preference-based alignment Contrastive Preference Optimization Supervised Fine-Tuning alignment metric lexical metrics