title: Early-Exit and Instant Confidence Translation Quality Estimation
summary: Quality estimation is omnipresent in machine translation, for both evaluation
  and generation. Unfortunately, quality estimation models are often opaque and
  computationally expensive, making them impractical to be part of large-scale
  pipelines. In this work, we tackle two connected challenges: (1) reducing the
  cost of quality estimation at scale, and (2) developing an inexpensive
  uncertainty estimation method for quality estimation. To address the latter, we
  introduce Instant Confidence COMET, an uncertainty-aware quality estimation
  model that matches the performance of previous approaches at a fraction of
  their costs. We extend this to Early-Exit COMET, a quality estimation model
  that can compute quality scores and associated confidences already at early
  model layers, allowing us to early-exit computations and reduce evaluation
  costs. We also apply our model to machine translation reranking. We combine
  Early-Exit COMET with an upper confidence bound bandit algorithm to find the
  best candidate from a large pool without having to run the full evaluation
  model on all candidates. In both cases (evaluation and reranking) our methods
  reduce the required compute by 50% with very little degradation in performance.
paper_authors._id: 67b835f98512a3eca052c0ee
paper_authors.name: Vilém Zouhar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T14:09:46.337Z
paper_authors._id: 6304ece07424d937fa35fb98
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6304ece07424d937fa35fb98/6qZoqm-Ti8CiDcCHEt1sE.jpeg
paper_authors.fullname: Vilém Zouhar
paper_authors.type: user
paper_authors.user: zouharvi
paper_authors._id: 67b835f98512a3eca052c0ef
paper_authors.name: Maike Züfle
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:38:17.623Z
paper_authors._id: 6552004b4d9e71e17b35fa0b
paper_authors.avatarUrl: /avatars/341ac1551588d5fcf5f3526fc06ff702.svg
paper_authors.fullname: Maike Züfle
paper_authors.type: user
paper_authors.user: maikez
paper_authors._id: 67b835f98512a3eca052c0f0
paper_authors.name: Beni Egressy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:38:24.223Z
paper_authors._id: 6735d61d0c6b2cc068fe7cda
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OD07myJrBbPaM1BUknLP-.png
paper_authors.fullname: Beni Egressy
paper_authors.type: user
paper_authors.user: egressbi
paper_authors._id: 67b835f98512a3eca052c0f1
paper_authors.name: Julius Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:38:33.820Z
paper_authors._id: 641af90d1911d3be6742928a
paper_authors.avatarUrl: /avatars/d6d34c2d49cb8dfdccc6681aacf47cd4.svg
paper_authors.fullname: Julius Cheng
paper_authors.type: user
paper_authors.user: juliuscheng
paper_authors._id: 67b835f98512a3eca052c0f2
paper_authors.name: Jan Niehues
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:38:40.110Z
paper_authors._id: 63394998474cfeb1a85bde3f
paper_authors.avatarUrl: /avatars/43d997eb118e6c6da9604e0c0bf0e63e.svg
paper_authors.fullname: Jan Niehues
paper_authors.type: user
paper_authors.user: jannieh
paper_title: Early-Exit and Instant Confidence Translation Quality Estimation
paper_summary: Quality estimation is omnipresent in machine translation, for both evaluation
  and generation. Unfortunately, quality estimation models are often opaque and
  computationally expensive, making them impractical to be part of large-scale
  pipelines. In this work, we tackle two connected challenges: (1) reducing the
  cost of quality estimation at scale, and (2) developing an inexpensive
  uncertainty estimation method for quality estimation. To address the latter, we
  introduce Instant Confidence COMET, an uncertainty-aware quality estimation
  model that matches the performance of previous approaches at a fraction of
  their costs. We extend this to Early-Exit COMET, a quality estimation model
  that can compute quality scores and associated confidences already at early
  model layers, allowing us to early-exit computations and reduce evaluation
  costs. We also apply our model to machine translation reranking. We combine
  Early-Exit COMET with an upper confidence bound bandit algorithm to find the
  best candidate from a large pool without having to run the full evaluation
  model on all candidates. In both cases (evaluation and reranking) our methods
  reduce the required compute by 50% with very little degradation in performance.
paper_ai_summary: The paper presents lightweight quality and uncertainty estimation models for machine translation that significantly reduce computational costs while maintaining performance.
paper_ai_keywords: quality estimation machine translation uncertainty estimation instant confidence comet early-exit comet upper confidence bound bandit algorithm