title: Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient
    Pruning of A Multilingual ASR Model
summary: Neural network pruning offers an effective method for compressing a
  multilingual automatic speech recognition (ASR) model with minimal performance
  loss. However, it entails several rounds of pruning and re-training needed to
  be run for each language. In this work, we propose the use of an adaptive
  masking approach in two scenarios for pruning a multilingual ASR model
  efficiently, each resulting in sparse monolingual models or a sparse
  multilingual model (named as Dynamic ASR Pathways). Our approach dynamically
  adapts the sub-network, avoiding premature decisions about a fixed sub-network
  structure. We show that our approach outperforms existing pruning methods when
  targeting sparse monolingual models. Further, we illustrate that Dynamic ASR
  Pathways jointly discovers and trains better sub-networks (pathways) of a
  single multilingual model by adapting from different sub-network
  initializations, thereby reducing the need for language-specific pruning.
paper_authors._id: 6511153753b1e2d59e4f1b5f
paper_authors.name: Jiamin Xie
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-09-25T15:49:00.029Z
paper_authors._id: 6511abcf44ab66c44fe2a2ad
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6511abcf44ab66c44fe2a2ad/_QPAByxPGxRQa0OFLgiys.jpeg
paper_authors.fullname: Jiamin Xie
paper_authors.type: user
paper_authors.user: jiaminx-hug
paper_authors._id: 6511153753b1e2d59e4f1b60
paper_authors.name: Ke Li
paper_authors._id: 6511153753b1e2d59e4f1b61
paper_authors.name: Jinxi Guo
paper_authors._id: 6511153753b1e2d59e4f1b62
paper_authors.name: Andros Tjandra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T11:07:20.169Z
paper_authors._id: 647779c6c21496284db45835
paper_authors.avatarUrl: /avatars/c1176e57f5d0aaa5f50bbde48a19af01.svg
paper_authors.fullname: Andros Tjandra
paper_authors.type: user
paper_authors.user: androstj
paper_authors._id: 6511153753b1e2d59e4f1b63
paper_authors.name: Yuan Shangguan
paper_authors._id: 6511153753b1e2d59e4f1b64
paper_authors.name: Leda Sari
paper_authors._id: 6511153753b1e2d59e4f1b65
paper_authors.name: Chunyang Wu
paper_authors._id: 6511153753b1e2d59e4f1b66
paper_authors.name: Junteng Jia
paper_authors._id: 6511153753b1e2d59e4f1b67
paper_authors.name: Jay Mahadeokar
paper_authors._id: 6511153753b1e2d59e4f1b68
paper_authors.name: Ozlem Kalinli
paper_title: Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient
    Pruning of A Multilingual ASR Model
paper_summary: Neural network pruning offers an effective method for compressing a
  multilingual automatic speech recognition (ASR) model with minimal performance
  loss. However, it entails several rounds of pruning and re-training needed to
  be run for each language. In this work, we propose the use of an adaptive
  masking approach in two scenarios for pruning a multilingual ASR model
  efficiently, each resulting in sparse monolingual models or a sparse
  multilingual model (named as Dynamic ASR Pathways). Our approach dynamically
  adapts the sub-network, avoiding premature decisions about a fixed sub-network
  structure. We show that our approach outperforms existing pruning methods when
  targeting sparse monolingual models. Further, we illustrate that Dynamic ASR
  Pathways jointly discovers and trains better sub-networks (pathways) of a
  single multilingual model by adapting from different sub-network
  initializations, thereby reducing the need for language-specific pruning.
paper_ai_summary: An adaptive masking approach efficiently prunes multilingual ASR models, outperforming existing methods and reducing language-specific pruning need.
paper_ai_keywords: neural network pruning multilingual automatic speech recognition adaptive masking sparse monolingual models sparse multilingual model Dynamic ASR Pathways sub-network adaptation