title: Accelerating Transducers through Adjacent Token Merging
summary: Recent end-to-end automatic speech recognition (ASR) systems often utilize a
  Transformer-based acoustic encoder that generates embedding at a high frame
  rate. However, this design is inefficient, particularly for long speech signals
  due to the quadratic computation of self-attention. To address this, we propose
  a new method, Adjacent Token Merging (A-ToMe), which gradually combines
  adjacent tokens with high similarity scores between their key values. In this
  way, the total time step could be reduced, and the inference of both the
  encoder and joint network is accelerated. Experiments on LibriSpeech show that
  our method can reduce 57% of tokens and improve the inference speed on GPU by
  70% without any notable loss of accuracy. Additionally, we demonstrate that
  A-ToMe is also an effective solution to reduce tokens in long-form ASR, where
  the input speech consists of multiple utterances.
paper_authors._id: 649cf693bc919c0fd744d2c0
paper_authors.name: Yuang Li
paper_authors._id: 649cf693bc919c0fd744d2c1
paper_authors.name: Yu Wu
paper_authors._id: 649cf693bc919c0fd744d2c2
paper_authors.name: Jinyu Li
paper_authors._id: 649cf693bc919c0fd744d2c3
paper_authors.name: Shujie Liu
paper_title: Accelerating Transducers through Adjacent Token Merging
paper_summary: Recent end-to-end automatic speech recognition (ASR) systems often utilize a
  Transformer-based acoustic encoder that generates embedding at a high frame
  rate. However, this design is inefficient, particularly for long speech signals
  due to the quadratic computation of self-attention. To address this, we propose
  a new method, Adjacent Token Merging (A-ToMe), which gradually combines
  adjacent tokens with high similarity scores between their key values. In this
  way, the total time step could be reduced, and the inference of both the
  encoder and joint network is accelerated. Experiments on LibriSpeech show that
  our method can reduce 57% of tokens and improve the inference speed on GPU by
  70% without any notable loss of accuracy. Additionally, we demonstrate that
  A-ToMe is also an effective solution to reduce tokens in long-form ASR, where
  the input speech consists of multiple utterances.
paper_ai_summary: A-ToMe reduces computational costs in ASR systems by merging adjacent tokens with high similarity, thereby accelerating inference without sacrificing accuracy.
paper_ai_keywords: Transformer-based acoustic encoder self-attention adjacent token merging A-ToMe LibriSpeech long-form ASR