ProGRes: Prompted Generative Rescoring on ASR n-Best
<|reference_start|>ProGRes: Prompted Generative Rescoring on ASR n-Best: Large Language Models (LLMs) have shown their ability to improve the performance of speech recognizers by effectively rescoring the n-best hypotheses generated during the beam search process. However, the best way to exploit recent generative instruction-tuned LLMs for hypothesis rescoring is still unclear. This paper proposes a novel method that uses instruction-tuned LLMs to dynamically expand the n-best speech recognition hypotheses with new hypotheses generated through appropriately-prompted LLMs. Specifically, we introduce a new zero-shot method for ASR n-best rescoring, which combines confidence scores, LLM sequence scoring, and prompt-based hypothesis generation. We compare Llama-3-Instruct, GPT-3.5 Turbo, and GPT-4 Turbo as prompt-based generators with Llama-3 as sequence scorer LLM. We evaluated our approach using different speech recognizers and observed significant relative improvement in the word error rate (WER) ranging from 5% to 25%.<|reference_end|>
@article{tur2024progres:,
    title={ProGRes: Prompted Generative Rescoring on ASR n-Best},
    author={Ada Defne Tur, Adel Moumen, Mirco Ravanelli},
    journal={arXiv preprint arXiv:2409.00217},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.00217},
    primaryClass={cs.CL cs.SD eess.AS}
}