title: NeKo: Toward Post Recognition Generative Correction Large Language
    Models with Task-Oriented Experts
summary: Construction of a general-purpose post-recognition error corrector poses a
  crucial question: how can we most effectively train a model on a large mixture
  of domain datasets? The answer would lie in learning dataset-specific features
  and digesting their knowledge in a single model. Previous methods achieve this
  by having separate correction language models, resulting in a significant
  increase in parameters. In this work, we present Mixture-of-Experts as a
  solution, highlighting that MoEs are much more than a scalability tool. We
  propose a Multi-Task Correction MoE, where we train the experts to become an
  ``expert'' of speech-to-text, language-to-text and vision-to-text datasets by
  learning to route each dataset's tokens to its mapped expert. Experiments on
  the Open ASR Leaderboard show that we explore a new state-of-the-art
  performance by achieving an average relative 5.0% WER reduction and
  substantial improvements in BLEU scores for speech and translation tasks. On
  zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-Opus with 15.5% to
  27.6% relative WER reduction in the Hyporadise benchmark. NeKo performs
  competitively on grammar and post-OCR correction as a multi-task model.
paper_authors._id: 67335d4cd40c698f6d001c1f
paper_authors.name: Yen-Ting Lin
paper_authors._id: 67335d4cd40c698f6d001c20
paper_authors.name: Chao-Han Huck Yang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-12T13:51:09.350Z
paper_authors._id: 629e1b71bb6419817ed7566c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/629e1b71bb6419817ed7566c/0ZCt-11eQtRDCOk9AozOp.jpeg
paper_authors.fullname: Huck Yang
paper_authors.type: user
paper_authors.user: huckiyang
paper_authors._id: 67335d4cd40c698f6d001c21
paper_authors.name: Zhehuai Chen
paper_authors._id: 67335d4cd40c698f6d001c22
paper_authors.name: Piotr Zelasko
paper_authors._id: 67335d4cd40c698f6d001c23
paper_authors.name: Xuesong Yang
paper_authors._id: 67335d4cd40c698f6d001c24
paper_authors.name: Zih-Ching Chen
paper_authors._id: 67335d4cd40c698f6d001c25
paper_authors.name: Krishna C Puvvada
paper_authors._id: 67335d4cd40c698f6d001c26
paper_authors.name: Szu-Wei Fu
paper_authors._id: 67335d4cd40c698f6d001c27
paper_authors.name: Ke Hu
paper_authors._id: 67335d4cd40c698f6d001c28
paper_authors.name: Jun Wei Chiu
paper_authors._id: 67335d4cd40c698f6d001c29
paper_authors.name: Jagadeesh Balam
paper_authors._id: 67335d4cd40c698f6d001c2a
paper_authors.name: Boris Ginsburg
paper_authors._id: 67335d4cd40c698f6d001c2b
paper_authors.name: Yu-Chiang Frank Wang
paper_title: NeKo: Toward Post Recognition Generative Correction Large Language
    Models with Task-Oriented Experts
paper_summary: Construction of a general-purpose post-recognition error corrector poses a
  crucial question: how can we most effectively train a model on a large mixture
  of domain datasets? The answer would lie in learning dataset-specific features
  and digesting their knowledge in a single model. Previous methods achieve this
  by having separate correction language models, resulting in a significant
  increase in parameters. In this work, we present Mixture-of-Experts as a
  solution, highlighting that MoEs are much more than a scalability tool. We
  propose a Multi-Task Correction MoE, where we train the experts to become an
  ``expert'' of speech-to-text, language-to-text and vision-to-text datasets by
  learning to route each dataset's tokens to its mapped expert. Experiments on
  the Open ASR Leaderboard show that we explore a new state-of-the-art
  performance by achieving an average relative 5.0% WER reduction and
  substantial improvements in BLEU scores for speech and translation tasks. On
  zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-Opus with 15.5% to
  27.6% relative WER reduction in the Hyporadise benchmark. NeKo performs
  competitively on grammar and post-OCR correction as a multi-task model.
paper_ai_summary: A Multi-Task Correction MoE enhances error correction across different domains, achieving state-of-the-art performance with improved WER and BLEU scores.
paper_ai_keywords: Mixture-of-Experts Multi-Task Correction MoE speech-to-text language-to-text vision-to-text Open ASR Leaderboard WER BLEU zero-shot evaluation NeKo GPT-3.5 Claude-Opus Hyporadise post-OCR correction