Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni,
on an audio question answering dataset with the reinforcement learning method
GRPO. This leads to new State-of-the-Art performance on the recent MMAU
benchmark. Omni-R1 achieves the highest accuracies on the sounds, music,
speech, and overall average categories, both on the Test-mini and Test-full
splits. To understand the performance improvement, we tested models both with
and without audio and found that much of the performance improvement from GRPO
could be attributed to better text-based reasoning. We also made a surprising
discovery that fine-tuning without audio on a text-only dataset was effective
at improving the audio-based performance.
6825cfe4658e0204a1b492c7
Andrew Rouditchenko
extracted_confirmed
2025-05-15T11:29:56.341Z
6253482bde8d23c3ce8816d0
/avatars/44475ac0e514a4e88d148ab43d0fd489.svg
Andrew Rouditchenko
user
h9LtLSb
6825cfe4658e0204a1b492c8
Saurabhchand Bhati
claimed_verified
2025-06-23T08:32:45.923Z
67d301cbba86f5d66eb73d7c
/avatars/8546bbd2145c16d4be5675624516b649.svg
Saurabhchand Bhati
user
saurabhati
6825cfe4658e0204a1b492c9
Edson Araujo
6825cfe4658e0204a1b492ca
Samuel Thomas
6825cfe4658e0204a1b492cb
Hilde Kuehne
6825cfe4658e0204a1b492cc
Rogerio Feris
6825cfe4658e0204a1b492cd
James Glass
Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni,
on an audio question answering dataset with the reinforcement learning method
GRPO. This leads to new State-of-the-Art performance on the recent MMAU
benchmark. Omni-R1 achieves the highest accuracies on the sounds, music,
speech, and overall average categories, both on the Test-mini and Test-full
splits. To understand the performance improvement, we tested models both with
and without audio and found that much of the performance improvement from GRPO
could be attributed to better text-based reasoning. We also made a surprising
discovery that fine-tuning without audio on a text-only dataset was effective
at improving the audio-based performance.
Omni-R1 fine-tunes Qwen2.5-Omni with GRPO on an audio QA dataset, achieving state-of-the-art performance in sound, music, speech, and average categories.
multi-modal LLM Qwen2.5-Omni audio question answering reinforcement learning GRPO MMAU benchmark text-based reasoning