Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models
<|reference_start|>Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models: This paper explores speculative speech recognition (SSR), where we empower conventional automatic speech recognition (ASR) with speculation capabilities, allowing the recognizer to run ahead of audio. We introduce a metric for measuring SSR performance and we propose a model which does SSR by combining a RNN-Transducer-based ASR system with an audio-prefixed language model (LM). The ASR system transcribes ongoing audio and feeds the resulting transcripts, along with an audio-dependent prefix, to the LM, which speculates likely completions for the transcriptions. We experiment with a variety of ASR datasets on which show the efficacy our method and the feasibility of SSR as a method of reducing ASR latency.<|reference_end|>
@article{yusuf2024speculative,
    title={Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of
  Language Models},
    author={Bolaji Yusuf, Murali Karthick Baskar, Andrew Rosenberg, Bhuvana
  Ramabhadran},
    journal={arXiv preprint arXiv:2407.04641},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.04641},
    primaryClass={eess.AS cs.CL}
}