title: In-Context Learning Boosts Speech Recognition via Human-like Adaptation
    to Speakers and Language Varieties
summary: Human listeners readily adjust to unfamiliar speakers and language varieties
  through exposure, but do these adaptation benefits extend to state-of-the-art
  spoken language models? We introduce a scalable framework that allows for
  in-context learning (ICL) in Phi-4 Multimodal using interleaved task prompts
  and audio-text pairs, and find that as few as 12 example utterances (~50
  seconds) at inference time reduce word error rates by a relative 19.7% (1.2
  pp.) on average across diverse English corpora. These improvements are most
  pronounced in low-resource varieties, when the context and target speaker
  match, and when more examples are provided--though scaling our procedure yields
  diminishing marginal returns to context length. Overall, we find that our novel
  ICL adaptation scheme (1) reveals a similar performance profile to human
  listeners, and (2) demonstrates consistent improvements to automatic speech
  recognition (ASR) robustness across diverse speakers and language backgrounds.
  While adaptation succeeds broadly, significant gaps remain for certain
  varieties, revealing where current models still fall short of human
  flexibility. We release our prompts and code on GitHub.
paper_authors._id: 682fcd1de495b6677027fe85
paper_authors.name: Nathan Roll
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:16:56.162Z
paper_authors._id: 631395b2e90d5240127ca48f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1673106093213-631395b2e90d5240127ca48f.png
paper_authors.fullname: Nathan Roll
paper_authors.type: user
paper_authors.user: NathanRoll
paper_authors._id: 682fcd1de495b6677027fe86
paper_authors.name: Calbert Graham
paper_authors._id: 682fcd1de495b6677027fe87
paper_authors.name: Yuka Tatsumi
paper_authors._id: 682fcd1de495b6677027fe88
paper_authors.name: Kim Tien Nguyen
paper_authors._id: 682fcd1de495b6677027fe89
paper_authors.name: Meghan Sumner
paper_authors._id: 682fcd1de495b6677027fe8a
paper_authors.name: Dan Jurafsky
paper_title: In-Context Learning Boosts Speech Recognition via Human-like Adaptation
    to Speakers and Language Varieties
paper_summary: Human listeners readily adjust to unfamiliar speakers and language varieties
  through exposure, but do these adaptation benefits extend to state-of-the-art
  spoken language models? We introduce a scalable framework that allows for
  in-context learning (ICL) in Phi-4 Multimodal using interleaved task prompts
  and audio-text pairs, and find that as few as 12 example utterances (~50
  seconds) at inference time reduce word error rates by a relative 19.7% (1.2
  pp.) on average across diverse English corpora. These improvements are most
  pronounced in low-resource varieties, when the context and target speaker
  match, and when more examples are provided--though scaling our procedure yields
  diminishing marginal returns to context length. Overall, we find that our novel
  ICL adaptation scheme (1) reveals a similar performance profile to human
  listeners, and (2) demonstrates consistent improvements to automatic speech
  recognition (ASR) robustness across diverse speakers and language backgrounds.
  While adaptation succeeds broadly, significant gaps remain for certain
  varieties, revealing where current models still fall short of human
  flexibility. We release our prompts and code on GitHub.
paper_ai_summary: In-context learning in Phi-4 Multimodal demonstrates significant improvements in automatic speech recognition robustness with a small number of example utterances, showing a performance profile similar to human listeners.
paper_ai_keywords: in-context learning ICL Phi-4 Multimodal interleaved task prompts audio-text pairs word error rates automatic speech recognition ASR