SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based
  on Speech and Audio Information
Large audio-language models (LALMs) extend the large language models with
multimodal understanding in speech, audio, etc. While their performances on
speech and audio-processing tasks are extensively studied, their reasoning
abilities remain underexplored. Particularly, their multi-hop reasoning, the
ability to recall and integrate multiple facts, lacks systematic evaluation.
Existing benchmarks focus on general speech and audio-processing tasks,
conversational abilities, and fairness but overlook this aspect. To bridge this
gap, we introduce SAKURA, a benchmark assessing LALMs' multi-hop reasoning
based on speech and audio information. Results show that LALMs struggle to
integrate speech/audio representations for multi-hop reasoning, even when they
extract the relevant information correctly, highlighting a fundamental
challenge in multimodal reasoning. Our findings expose a critical limitation in
LALMs, offering insights and resources for future research.
6830741133a25ec4e8a40563
Chih-Kai Yang
extracted_confirmed
2025-05-23T13:21:30.080Z
646fa3016441111304fec68d
/avatars/923629340f3785ae8c6e52cf3674d5c2.svg
Chih-Kai Yang
user
zenyn
6830741133a25ec4e8a40564
Neo Ho
6830741133a25ec4e8a40565
Yen-Ting Piao
claimed_verified
2025-06-13T14:59:49.049Z
6623864025bb4e9dc1f09695
/avatars/d37dc450d48cd1e9be36608c0ece0bcd.svg
Yenting
user
yenting-biao
6830741133a25ec4e8a40566
Hung-yi Lee
SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based
  on Speech and Audio Information
Large audio-language models (LALMs) extend the large language models with
multimodal understanding in speech, audio, etc. While their performances on
speech and audio-processing tasks are extensively studied, their reasoning
abilities remain underexplored. Particularly, their multi-hop reasoning, the
ability to recall and integrate multiple facts, lacks systematic evaluation.
Existing benchmarks focus on general speech and audio-processing tasks,
conversational abilities, and fairness but overlook this aspect. To bridge this
gap, we introduce SAKURA, a benchmark assessing LALMs' multi-hop reasoning
based on speech and audio information. Results show that LALMs struggle to
integrate speech/audio representations for multi-hop reasoning, even when they
extract the relevant information correctly, highlighting a fundamental
challenge in multimodal reasoning. Our findings expose a critical limitation in
LALMs, offering insights and resources for future research.
SAKURA is introduced to evaluate the multi-hop reasoning abilities of large audio-language models, revealing their struggles in integrating speech/audio representations.
large audio-language models multimodal understanding speech audio multi-hop reasoning SAKURA