title: OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and
    Cleaning
summary: The Open Whisper-style Speech Models (OWSM) project has developed a series of
  fully open speech foundation models using academic-scale resources, but their
  training data remains insufficient. This work enhances OWSM by integrating
  YODAS, a large-scale web-crawled dataset with a Creative Commons license.
  However, incorporating YODAS is nontrivial due to its wild nature, which
  introduces challenges such as incorrect language labels and audio-text
  misalignments. To address this, we develop a scalable data-cleaning pipeline
  using public toolkits, yielding a dataset with 166,000 hours of speech across
  75 languages. Our new series of OWSM v4 models, trained on this curated dataset
  alongside existing OWSM data, significantly outperform previous versions on
  multilingual benchmarks. Our models even match or surpass frontier industrial
  models like Whisper and MMS in multiple scenarios. We will publicly release the
  cleaned YODAS data, pre-trained models, and all associated scripts via the
  ESPnet toolkit.
paper_authors._id: 683e64eb3e5a54d05365ddc6
paper_authors.name: Yifan Peng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:46:09.029Z
paper_authors._id: 61809f31a367a8f5351ef353
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61809f31a367a8f5351ef353/s5eQ00YeoirakzE_rJ0cy.jpeg
paper_authors.fullname: Yifan Peng
paper_authors.type: user
paper_authors.user: pyf98
paper_authors._id: 683e64eb3e5a54d05365ddc7
paper_authors.name: Shakeel Muhammad
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:03:08.892Z
paper_authors._id: 657821170766f274f3c9b35f
paper_authors.avatarUrl: /avatars/3e95b1fb9578ec27d9ad176c34b979c4.svg
paper_authors.fullname: Muhammad Shakeel
paper_authors.type: user
paper_authors.user: shakeel1
paper_authors._id: 683e64eb3e5a54d05365ddc8
paper_authors.name: Yui Sudo
paper_authors._id: 683e64eb3e5a54d05365ddc9
paper_authors.name: William Chen
paper_authors._id: 683e64eb3e5a54d05365ddca
paper_authors.name: Jinchuan Tian
paper_authors._id: 683e64eb3e5a54d05365ddcb
paper_authors.name: Chyi-Jiunn Lin
paper_authors._id: 683e64eb3e5a54d05365ddcc
paper_authors.name: Shinji Watanabe
paper_title: OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and
    Cleaning
paper_summary: The Open Whisper-style Speech Models (OWSM) project has developed a series of
  fully open speech foundation models using academic-scale resources, but their
  training data remains insufficient. This work enhances OWSM by integrating
  YODAS, a large-scale web-crawled dataset with a Creative Commons license.
  However, incorporating YODAS is nontrivial due to its wild nature, which
  introduces challenges such as incorrect language labels and audio-text
  misalignments. To address this, we develop a scalable data-cleaning pipeline
  using public toolkits, yielding a dataset with 166,000 hours of speech across
  75 languages. Our new series of OWSM v4 models, trained on this curated dataset
  alongside existing OWSM data, significantly outperform previous versions on
  multilingual benchmarks. Our models even match or surpass frontier industrial
  models like Whisper and MMS in multiple scenarios. We will publicly release the
  cleaned YODAS data, pre-trained models, and all associated scripts via the
  ESPnet toolkit.
paper_ai_summary: The OWSM project is enhanced with a large-scale, cleaned web dataset, leading to improved multilingual speech models comparable to leading industrial models.
paper_ai_keywords: speach foundation models YODAS data-cleaning pipeline multilingual benchmarks Whisper MMS ESPnet toolkit