WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered
significant attention in the speech domain. However, the evaluation of spoken
dialogue models' conversational performance has largely been overlooked. This
is primarily due to the intelligent chatbots convey a wealth of non-textual
information which cannot be easily measured using text-based language models
like ChatGPT. To address this gap, we propose WavReward, a reward feedback
model based on audio language models that can evaluate both the IQ and EQ of
spoken dialogue systems with speech input. Specifically, 1) based on audio
language models, WavReward incorporates the deep reasoning process and the
nonlinear reward mechanism for post-training. By utilizing multi-sample
feedback via the reinforcement learning algorithm, we construct a specialized
evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a
preference dataset used to train WavReward. ChatReward-30K includes both
comprehension and generation aspects of spoken dialogue models. These scenarios
span various tasks, such as text-based chats, nine acoustic attributes of
instruction chats, and implicit chats. WavReward outperforms previous
state-of-the-art evaluation models across multiple spoken dialogue scenarios,
achieving a substantial improvement about Qwen2.5-Omni in objective accuracy
from 55.1% to 91.5%. In subjective A/B testing, WavReward also leads by a
margin of 83%. Comprehensive ablation studies confirm the necessity of each
component of WavReward. All data and code will be publicly at
https://github.com/jishengpeng/WavReward after the paper is accepted.
682617fc0b576b6ed65cedb7
Shengpeng Ji
claimed_verified
2025-05-16T07:12:18.381Z
65afc920091a8ca32b1b0e96
https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg
jishengpeng
user
novateur
682617fc0b576b6ed65cedb8
Tianle Liang
682617fc0b576b6ed65cedb9
Yangzhuo Li
682617fc0b576b6ed65cedba
Jialong Zuo
682617fc0b576b6ed65cedbb
Minghui Fang
682617fc0b576b6ed65cedbc
Jinzheng He
682617fc0b576b6ed65cedbd
Yifu Chen
682617fc0b576b6ed65cedbe
Zhengqing Liu
682617fc0b576b6ed65cedbf
Ziyue Jiang
682617fc0b576b6ed65cedc0
Xize Cheng
682617fc0b576b6ed65cedc1
Siqi Zheng
682617fc0b576b6ed65cedc2
Jin Xu
682617fc0b576b6ed65cedc3
Junyang Lin
682617fc0b576b6ed65cedc4
Zhou Zhao
WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered
significant attention in the speech domain. However, the evaluation of spoken
dialogue models' conversational performance has largely been overlooked. This
is primarily due to the intelligent chatbots convey a wealth of non-textual
information which cannot be easily measured using text-based language models
like ChatGPT. To address this gap, we propose WavReward, a reward feedback
model based on audio language models that can evaluate both the IQ and EQ of
spoken dialogue systems with speech input. Specifically, 1) based on audio
language models, WavReward incorporates the deep reasoning process and the
nonlinear reward mechanism for post-training. By utilizing multi-sample
feedback via the reinforcement learning algorithm, we construct a specialized
evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a
preference dataset used to train WavReward. ChatReward-30K includes both
comprehension and generation aspects of spoken dialogue models. These scenarios
span various tasks, such as text-based chats, nine acoustic attributes of
instruction chats, and implicit chats. WavReward outperforms previous
state-of-the-art evaluation models across multiple spoken dialogue scenarios,
achieving a substantial improvement about Qwen2.5-Omni in objective accuracy
from 55.1% to 91.5%. In subjective A/B testing, WavReward also leads by a
margin of 83%. Comprehensive ablation studies confirm the necessity of each
component of WavReward. All data and code will be publicly at
https://github.com/jishengpeng/WavReward after the paper is accepted.
WavReward, a reward feedback model based on audio language models, evaluates the conversational performance of spoken dialogue systems, addressing the gap in evaluation by incorporating deep reasoning and nonlinear rewards.
audio language models deep reasoning nonlinear reward mechanism reinforcement learning multi-sample feedback spoken dialogue systems preference dataset comprehensive ablation studies