title: WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
summary: End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered
  significant attention in the speech domain. However, the evaluation of spoken
  dialogue models' conversational performance has largely been overlooked. This
  is primarily due to the intelligent chatbots convey a wealth of non-textual
  information which cannot be easily measured using text-based language models
  like ChatGPT. To address this gap, we propose WavReward, a reward feedback
  model based on audio language models that can evaluate both the IQ and EQ of
  spoken dialogue systems with speech input. Specifically, 1) based on audio
  language models, WavReward incorporates the deep reasoning process and the
  nonlinear reward mechanism for post-training. By utilizing multi-sample
  feedback via the reinforcement learning algorithm, we construct a specialized
  evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a
  preference dataset used to train WavReward. ChatReward-30K includes both
  comprehension and generation aspects of spoken dialogue models. These scenarios
  span various tasks, such as text-based chats, nine acoustic attributes of
  instruction chats, and implicit chats. WavReward outperforms previous
  state-of-the-art evaluation models across multiple spoken dialogue scenarios,
  achieving a substantial improvement about Qwen2.5-Omni in objective accuracy
  from 55.1% to 91.5%. In subjective A/B testing, WavReward also leads by a
  margin of 83%. Comprehensive ablation studies confirm the necessity of each
  component of WavReward. All data and code will be publicly at
  https://github.com/jishengpeng/WavReward after the paper is accepted.
paper_authors._id: 682617fc0b576b6ed65cedb7
paper_authors.name: Shengpeng Ji
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-16T07:12:18.381Z
paper_authors._id: 65afc920091a8ca32b1b0e96
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg
paper_authors.fullname: jishengpeng
paper_authors.type: user
paper_authors.user: novateur
paper_authors._id: 682617fc0b576b6ed65cedb8
paper_authors.name: Tianle Liang
paper_authors._id: 682617fc0b576b6ed65cedb9
paper_authors.name: Yangzhuo Li
paper_authors._id: 682617fc0b576b6ed65cedba
paper_authors.name: Jialong Zuo
paper_authors._id: 682617fc0b576b6ed65cedbb
paper_authors.name: Minghui Fang
paper_authors._id: 682617fc0b576b6ed65cedbc
paper_authors.name: Jinzheng He
paper_authors._id: 682617fc0b576b6ed65cedbd
paper_authors.name: Yifu Chen
paper_authors._id: 682617fc0b576b6ed65cedbe
paper_authors.name: Zhengqing Liu
paper_authors._id: 682617fc0b576b6ed65cedbf
paper_authors.name: Ziyue Jiang
paper_authors._id: 682617fc0b576b6ed65cedc0
paper_authors.name: Xize Cheng
paper_authors._id: 682617fc0b576b6ed65cedc1
paper_authors.name: Siqi Zheng
paper_authors._id: 682617fc0b576b6ed65cedc2
paper_authors.name: Jin Xu
paper_authors._id: 682617fc0b576b6ed65cedc3
paper_authors.name: Junyang Lin
paper_authors._id: 682617fc0b576b6ed65cedc4
paper_authors.name: Zhou Zhao
paper_title: WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
paper_summary: End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered
  significant attention in the speech domain. However, the evaluation of spoken
  dialogue models' conversational performance has largely been overlooked. This
  is primarily due to the intelligent chatbots convey a wealth of non-textual
  information which cannot be easily measured using text-based language models
  like ChatGPT. To address this gap, we propose WavReward, a reward feedback
  model based on audio language models that can evaluate both the IQ and EQ of
  spoken dialogue systems with speech input. Specifically, 1) based on audio
  language models, WavReward incorporates the deep reasoning process and the
  nonlinear reward mechanism for post-training. By utilizing multi-sample
  feedback via the reinforcement learning algorithm, we construct a specialized
  evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a
  preference dataset used to train WavReward. ChatReward-30K includes both
  comprehension and generation aspects of spoken dialogue models. These scenarios
  span various tasks, such as text-based chats, nine acoustic attributes of
  instruction chats, and implicit chats. WavReward outperforms previous
  state-of-the-art evaluation models across multiple spoken dialogue scenarios,
  achieving a substantial improvement about Qwen2.5-Omni in objective accuracy
  from 55.1% to 91.5%. In subjective A/B testing, WavReward also leads by a
  margin of 83%. Comprehensive ablation studies confirm the necessity of each
  component of WavReward. All data and code will be publicly at
  https://github.com/jishengpeng/WavReward after the paper is accepted.
paper_ai_summary: WavReward, a reward feedback model based on audio language models, evaluates the conversational performance of spoken dialogue systems, addressing the gap in evaluation by incorporating deep reasoning and nonlinear rewards.
paper_ai_keywords: audio language models deep reasoning nonlinear reward mechanism reinforcement learning multi-sample feedback spoken dialogue systems preference dataset comprehensive ablation studies