title: LiteASR: Efficient Automatic Speech Recognition with Low-Rank
    Approximation
summary: Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper,
  rely on deep encoder-decoder architectures, and their encoders are a critical
  bottleneck for efficient deployment due to high computational intensity. We
  introduce LiteASR, a low-rank compression scheme for ASR encoders that
  significantly reduces inference costs while maintaining transcription accuracy.
  Our approach leverages the strong low-rank properties observed in intermediate
  activations: by applying principal component analysis (PCA) with a small
  calibration dataset, we approximate linear transformations with a chain of
  low-rank matrix multiplications, and further optimize self-attention to work in
  the reduced dimension. Evaluation results show that our method can compress
  Whisper large-v3's encoder size by over 50%, matching Whisper medium's size
  with better transcription accuracy, thereby establishing a new Pareto-optimal
  frontier of efficiency and performance. The code of LiteASR is available at
  https://github.com/efeslab/LiteASR.
paper_authors._id: 67c516998d02783fa3a52dc8
paper_authors.name: Keisuke Kamahori
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-03T08:07:02.986Z
paper_authors._id: 6304ac1a412a1b9d381ca378
paper_authors.avatarUrl: /avatars/f4724eb5afc2a3b0e61e6da7bfa7be27.svg
paper_authors.fullname: Keisuke Kamahori
paper_authors.type: user
paper_authors.user: kamahori
paper_authors._id: 67c516998d02783fa3a52dc9
paper_authors.name: Jungo Kasai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:43:49.097Z
paper_authors._id: 62908273c740ebb981a6dba4
paper_authors.avatarUrl: /avatars/465f50369c367b07670f5209c83d65f2.svg
paper_authors.fullname: Jungo Kasai
paper_authors.type: user
paper_authors.user: jungok
paper_authors._id: 67c516998d02783fa3a52dca
paper_authors.name: Noriyuki Kojima
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:43:56.698Z
paper_authors._id: 628c26a8b80bb09700d6af86
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1653352051245-noauth.jpeg
paper_authors.fullname: Noriyuki Kojima
paper_authors.type: user
paper_authors.user: kojimano
paper_authors._id: 67c516998d02783fa3a52dcb
paper_authors.name: Baris Kasikci
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:44:04.084Z
paper_authors._id: 654132fe5a9a913c6c870e79
paper_authors.avatarUrl: /avatars/2f6807eddef1929c571977e9af35f952.svg
paper_authors.fullname: Baris Kasikci
paper_authors.type: user
paper_authors.user: kasikci
paper_title: LiteASR: Efficient Automatic Speech Recognition with Low-Rank
    Approximation
paper_summary: Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper,
  rely on deep encoder-decoder architectures, and their encoders are a critical
  bottleneck for efficient deployment due to high computational intensity. We
  introduce LiteASR, a low-rank compression scheme for ASR encoders that
  significantly reduces inference costs while maintaining transcription accuracy.
  Our approach leverages the strong low-rank properties observed in intermediate
  activations: by applying principal component analysis (PCA) with a small
  calibration dataset, we approximate linear transformations with a chain of
  low-rank matrix multiplications, and further optimize self-attention to work in
  the reduced dimension. Evaluation results show that our method can compress
  Whisper large-v3's encoder size by over 50%, matching Whisper medium's size
  with better transcription accuracy, thereby establishing a new Pareto-optimal
  frontier of efficiency and performance. The code of LiteASR is available at
  https://github.com/efeslab/LiteASR.
paper_ai_keywords: encoder-decoder architectures low-rank compression inference costs transcription accuracy principal component analysis (PCA) low-rank matrix multiplications self-attention reduced dimension Pareto-optimal frontier