title: OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on
    E-Branchformer
summary: Recent studies have advocated for fully open foundation models to promote
  transparency and open science. As an initial step, the Open Whisper-style
  Speech Model (OWSM) reproduced OpenAI's Whisper using publicly available data
  and open-source toolkits. With the aim of reproducing Whisper, the previous
  OWSM v1 through v3 models were still based on Transformer, which might lead to
  inferior performance compared to other state-of-the-art speech encoders. In
  this work, we aim to improve the performance and efficiency of OWSM without
  extra training data. We present E-Branchformer based OWSM v3.1 models at two
  scales, i.e., 100M and 1B. The 1B model is the largest E-Branchformer based
  speech model that has been made publicly available. It outperforms the previous
  OWSM v3 in a vast majority of evaluation benchmarks, while demonstrating up to
  25% faster inference speed. We publicly release the data preparation scripts,
  pre-trained models and training logs.
paper_authors._id: 65b9c9b05b16ab2f991e5cd9
paper_authors.name: Yifan Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:09:15.380Z
paper_authors._id: 61809f31a367a8f5351ef353
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61809f31a367a8f5351ef353/s5eQ00YeoirakzE_rJ0cy.jpeg
paper_authors.fullname: Yifan Peng
paper_authors.type: user
paper_authors.user: pyf98
paper_authors._id: 65b9c9b05b16ab2f991e5cda
paper_authors.name: Jinchuan Tian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:09:25.873Z
paper_authors._id: 64770ff6d7cf39f2e937b882
paper_authors.avatarUrl: /avatars/7b729501856f5073cf9e484811012f2a.svg
paper_authors.fullname: Jinchuan Tian
paper_authors.type: user
paper_authors.user: JinchuanTian
paper_authors._id: 65b9c9b05b16ab2f991e5cdb
paper_authors.name: William Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-25T07:34:35.865Z
paper_authors._id: 630438615c70c21d0eae6613
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630438615c70c21d0eae6613/KEHsB-znDIKHt7M0eyP8j.png
paper_authors.fullname: William Chen
paper_authors.type: user
paper_authors.user: wanchichen
paper_authors._id: 65b9c9b05b16ab2f991e5cdc
paper_authors.name: Siddhant Arora
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:09:59.498Z
paper_authors._id: 60d26d0d35274b8c1e4309f3
paper_authors.avatarUrl: /avatars/aa0810324729268eb4a9aca78c95a7f0.svg
paper_authors.fullname: Arora
paper_authors.type: user
paper_authors.user: Siddhant
paper_authors._id: 65b9c9b05b16ab2f991e5cdd
paper_authors.name: Brian Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:10:07.423Z
paper_authors._id: 620298a162b2b0e46e799698
paper_authors.avatarUrl: /avatars/ae7ecde8b706e7add741100270d7cbc8.svg
paper_authors.fullname: Brian Yan
paper_authors.type: user
paper_authors.user: brianyan918
paper_authors._id: 65b9c9b05b16ab2f991e5cde
paper_authors.name: Yui Sudo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:10:14.428Z
paper_authors._id: 6524aea9050781c16f324de5
paper_authors.avatarUrl: /avatars/06eecfb2e779ef88bc3c39fc50591831.svg
paper_authors.fullname: Yui Sudo
paper_authors.type: user
paper_authors.user: yuis0510
paper_authors._id: 65b9c9b05b16ab2f991e5cdf
paper_authors.name: Muhammad Shakeel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-14T07:07:11.384Z
paper_authors._id: 657821170766f274f3c9b35f
paper_authors.avatarUrl: /avatars/3e95b1fb9578ec27d9ad176c34b979c4.svg
paper_authors.fullname: Muhammad Shakeel
paper_authors.type: user
paper_authors.user: shakeel1
paper_authors._id: 65b9c9b05b16ab2f991e5ce0
paper_authors.name: Kwanghee Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:11:08.812Z
paper_authors._id: 64ba7c68710f5e5476877218
paper_authors.avatarUrl: /avatars/1e5ad96dd0b237db1bfee400aa8ed229.svg
paper_authors.fullname: Kwanghee Choi
paper_authors.type: user
paper_authors.user: juice500
paper_authors._id: 65b9c9b05b16ab2f991e5ce1
paper_authors.name: Jiatong Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:11:16.422Z
paper_authors._id: 6136ec7a40e43b8f748a0832
paper_authors.avatarUrl: /avatars/089afa82ecfe6432d2dbfe81048c6ffa.svg
paper_authors.fullname: Jiatong Shi
paper_authors.type: user
paper_authors.user: ftshijt
paper_authors._id: 65b9c9b05b16ab2f991e5ce2
paper_authors.name: Xuankai Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:11:23.253Z
paper_authors._id: 6136b5707c6d9b8e41469bec
paper_authors.avatarUrl: /avatars/5b7f4ee0ba36bb6b87accda0ee27bb17.svg
paper_authors.fullname: Xuankai Chang
paper_authors.type: user
paper_authors.user: simpleoier
paper_authors._id: 65b9c9b05b16ab2f991e5ce3
paper_authors.name: Jee-weon Jung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:11:29.586Z
paper_authors._id: 6226f40ee1cc4da2221557a0
paper_authors.avatarUrl: /avatars/9654850f5aeba1d7c19ae017096c78d7.svg
paper_authors.fullname: Jee-weon Jung
paper_authors.type: user
paper_authors.user: jungjee
paper_authors._id: 65b9c9b05b16ab2f991e5ce4
paper_authors.name: Shinji Watanabe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-31T10:11:51.264Z
paper_authors._id: 5feb9dbc29bcc7d2abb1cb76
paper_authors.avatarUrl: /avatars/908d392634377e674ba9e5a4318b101e.svg
paper_authors.fullname: Shinji Watanabe
paper_authors.type: user
paper_authors.user: sw005320
paper_title: OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on
    E-Branchformer
paper_summary: Recent studies have advocated for fully open foundation models to promote
  transparency and open science. As an initial step, the Open Whisper-style
  Speech Model (OWSM) reproduced OpenAI's Whisper using publicly available data
  and open-source toolkits. With the aim of reproducing Whisper, the previous
  OWSM v1 through v3 models were still based on Transformer, which might lead to
  inferior performance compared to other state-of-the-art speech encoders. In
  this work, we aim to improve the performance and efficiency of OWSM without
  extra training data. We present E-Branchformer based OWSM v3.1 models at two
  scales, i.e., 100M and 1B. The 1B model is the largest E-Branchformer based
  speech model that has been made publicly available. It outperforms the previous
  OWSM v3 in a vast majority of evaluation benchmarks, while demonstrating up to
  25% faster inference speed. We publicly release the data preparation scripts,
  pre-trained models and training logs.
paper_ai_summary: E-Branchformer-based OWSM v3.1 models improve speech processing efficiency and performance without additional training data.
paper_ai_keywords: Transformer E-Branchformer speech encoders data preparation scripts pre-trained models training logs