Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient
  Pruning of A Multilingual ASR Model
Neural network pruning offers an effective method for compressing a
multilingual automatic speech recognition (ASR) model with minimal performance
loss. However, it entails several rounds of pruning and re-training needed to
be run for each language. In this work, we propose the use of an adaptive
masking approach in two scenarios for pruning a multilingual ASR model
efficiently, each resulting in sparse monolingual models or a sparse
multilingual model (named as Dynamic ASR Pathways). Our approach dynamically
adapts the sub-network, avoiding premature decisions about a fixed sub-network
structure. We show that our approach outperforms existing pruning methods when
targeting sparse monolingual models. Further, we illustrate that Dynamic ASR
Pathways jointly discovers and trains better sub-networks (pathways) of a
single multilingual model by adapting from different sub-network
initializations, thereby reducing the need for language-specific pruning.
6511153753b1e2d59e4f1b5f
Jiamin Xie
extracted_pending
2023-09-25T15:49:00.029Z
6511abcf44ab66c44fe2a2ad
https://cdn-avatars.huggingface.co/v1/production/uploads/6511abcf44ab66c44fe2a2ad/_QPAByxPGxRQa0OFLgiys.jpeg
Jiamin Xie
user
jiaminx-hug
6511153753b1e2d59e4f1b60
Ke Li
6511153753b1e2d59e4f1b61
Jinxi Guo
6511153753b1e2d59e4f1b62
Andros Tjandra
admin_assigned
2023-09-25T11:07:20.169Z
647779c6c21496284db45835
/avatars/c1176e57f5d0aaa5f50bbde48a19af01.svg
Andros Tjandra
user
androstj
6511153753b1e2d59e4f1b63
Yuan Shangguan
6511153753b1e2d59e4f1b64
Leda Sari
6511153753b1e2d59e4f1b65
Chunyang Wu
6511153753b1e2d59e4f1b66
Junteng Jia
6511153753b1e2d59e4f1b67
Jay Mahadeokar
6511153753b1e2d59e4f1b68
Ozlem Kalinli
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient
  Pruning of A Multilingual ASR Model
Neural network pruning offers an effective method for compressing a
multilingual automatic speech recognition (ASR) model with minimal performance
loss. However, it entails several rounds of pruning and re-training needed to
be run for each language. In this work, we propose the use of an adaptive
masking approach in two scenarios for pruning a multilingual ASR model
efficiently, each resulting in sparse monolingual models or a sparse
multilingual model (named as Dynamic ASR Pathways). Our approach dynamically
adapts the sub-network, avoiding premature decisions about a fixed sub-network
structure. We show that our approach outperforms existing pruning methods when
targeting sparse monolingual models. Further, we illustrate that Dynamic ASR
Pathways jointly discovers and trains better sub-networks (pathways) of a
single multilingual model by adapting from different sub-network
initializations, thereby reducing the need for language-specific pruning.
An adaptive masking approach efficiently prunes multilingual ASR models, outperforming existing methods and reducing language-specific pruning need.
neural network pruning multilingual automatic speech recognition adaptive masking sparse monolingual models sparse multilingual model Dynamic ASR Pathways sub-network adaptation