title: StressTest: Can YOUR Speech LM Handle the Stress?
summary: Sentence stress refers to emphasis, placed on specific words within a spoken
  utterance to highlight or contrast an idea, or to introduce new information. It
  is often used to imply an underlying intention that is not explicitly stated.
  Recent advances in speech-aware language models (SLMs) have enabled direct
  processing of audio, allowing models to bypass transcription and access the
  full richness of the speech signal and perform audio reasoning tasks such as
  spoken question answering. Despite the crucial role of sentence stress in
  shaping meaning and speaker intent, it remains largely overlooked in evaluation
  and development of such models. In this work, we address this gap by
  introducing StressTest, a benchmark specifically designed to evaluate a model's
  ability to distinguish between interpretations of spoken sentences based on the
  stress pattern. We assess the performance of several leading SLMs and find
  that, despite their overall capabilities, they perform poorly on such tasks. To
  overcome this limitation, we propose a novel synthetic data generation
  pipeline, and create Stress17k, a training set that simulates change of meaning
  implied by stress variation. Then, we empirically show that optimizing models
  with this synthetic dataset aligns well with real-world recordings and enables
  effective finetuning of SLMs. Results suggest, that our finetuned model,
  StresSLM, significantly outperforms existing models on both sentence stress
  reasoning and detection tasks. Code, models, data, and audio samples -
  pages.cs.huji.ac.il/adiyoss-lab/stresstest.
paper_authors._id: 68394fb7f1e62a44f53ecf82
paper_authors.name: Iddo Yosha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:52:57.738Z
paper_authors._id: 658436f5c73f74776b19198a
paper_authors.avatarUrl: /avatars/3f1d76af6fc0405d663c9294318fe83e.svg
paper_authors.fullname: Iddo Yosha
paper_authors.type: user
paper_authors.user: iyosha
paper_authors._id: 68394fb7f1e62a44f53ecf83
paper_authors.name: Gallil Maimon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:52:41.076Z
paper_authors._id: 66b9bc2dacdbc1d0b39c3b50
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/hwR0pVfP_E8XjimXIxDOU.jpeg
paper_authors.fullname: Gallil Maimon
paper_authors.type: user
paper_authors.user: gallilmaimon
paper_authors._id: 68394fb7f1e62a44f53ecf84
paper_authors.name: Yossi Adi
paper_title: StressTest: Can YOUR Speech LM Handle the Stress?
paper_summary: Sentence stress refers to emphasis, placed on specific words within a spoken
  utterance to highlight or contrast an idea, or to introduce new information. It
  is often used to imply an underlying intention that is not explicitly stated.
  Recent advances in speech-aware language models (SLMs) have enabled direct
  processing of audio, allowing models to bypass transcription and access the
  full richness of the speech signal and perform audio reasoning tasks such as
  spoken question answering. Despite the crucial role of sentence stress in
  shaping meaning and speaker intent, it remains largely overlooked in evaluation
  and development of such models. In this work, we address this gap by
  introducing StressTest, a benchmark specifically designed to evaluate a model's
  ability to distinguish between interpretations of spoken sentences based on the
  stress pattern. We assess the performance of several leading SLMs and find
  that, despite their overall capabilities, they perform poorly on such tasks. To
  overcome this limitation, we propose a novel synthetic data generation
  pipeline, and create Stress17k, a training set that simulates change of meaning
  implied by stress variation. Then, we empirically show that optimizing models
  with this synthetic dataset aligns well with real-world recordings and enables
  effective finetuning of SLMs. Results suggest, that our finetuned model,
  StresSLM, significantly outperforms existing models on both sentence stress
  reasoning and detection tasks. Code, models, data, and audio samples -
  pages.cs.huji.ac.il/adiyoss-lab/stresstest.
paper_ai_summary: A StressTest benchmark and synthetic Stress17k dataset are introduced to improve speech-aware language models' ability to interpret sentence stress in spoken language.
paper_ai_keywords: speech-aware language models spoken question answering sentence stress benchmark synthetic data generation pipeline audio reasoning sentence stress reasoning and detection tasks