title: DiarizationLM: Speaker Diarization Post-Processing with Large Language
    Models
summary: In this paper, we introduce DiarizationLM, a framework to leverage large
  language models (LLM) to post-process the outputs from a speaker diarization
  system. Various goals can be achieved with the proposed framework, such as
  improving the readability of the diarized transcript, or reducing the word
  diarization error rate (WDER). In this framework, the outputs of the automatic
  speech recognition (ASR) and speaker diarization systems are represented as a
  compact textual format, which is included in the prompt to an optionally
  finetuned LLM. The outputs of the LLM can be used as the refined diarization
  results with the desired enhancement. As a post-processing step, this framework
  can be easily applied to any off-the-shelf ASR and speaker diarization systems
  without retraining existing components. Our experiments show that a finetuned
  PaLM 2-S model can reduce the WDER by rel. 25.9% on the Fisher telephone
  conversation dataset, and rel. 31% on the Callhome English dataset.
paper_authors._id: 659cc8d41d398a23816921bd
paper_authors.name: Quan Wang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-01-09T13:43:13.822Z
paper_authors._id: 64bea31d81caff7f184ad4a0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64bea31d81caff7f184ad4a0/hq4f77BRNoiY92musz9kq.jpeg
paper_authors.fullname: Quan
paper_authors.type: user
paper_authors.user: wq2012
paper_authors._id: 659cc8d41d398a23816921be
paper_authors.name: Yiling Huang
paper_authors._id: 659cc8d41d398a23816921bf
paper_authors.name: Guanlong Zhao
paper_authors._id: 659cc8d41d398a23816921c0
paper_authors.name: Evan Clark
paper_authors._id: 659cc8d41d398a23816921c1
paper_authors.name: Wei Xia
paper_authors._id: 659cc8d41d398a23816921c2
paper_authors.name: Hank Liao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:33:50.308Z
paper_authors._id: 655eeea1a74c042364de33e0
paper_authors.avatarUrl: /avatars/3249c16a373becf94bf13155b13d487e.svg
paper_authors.fullname: Hank Liao
paper_authors.type: user
paper_authors.user: HankLiao
paper_title: DiarizationLM: Speaker Diarization Post-Processing with Large Language
    Models
paper_summary: In this paper, we introduce DiarizationLM, a framework to leverage large
  language models (LLM) to post-process the outputs from a speaker diarization
  system. Various goals can be achieved with the proposed framework, such as
  improving the readability of the diarized transcript, or reducing the word
  diarization error rate (WDER). In this framework, the outputs of the automatic
  speech recognition (ASR) and speaker diarization systems are represented as a
  compact textual format, which is included in the prompt to an optionally
  finetuned LLM. The outputs of the LLM can be used as the refined diarization
  results with the desired enhancement. As a post-processing step, this framework
  can be easily applied to any off-the-shelf ASR and speaker diarization systems
  without retraining existing components. Our experiments show that a finetuned
  PaLM 2-S model can reduce the WDER by rel. 25.9% on the Fisher telephone
  conversation dataset, and rel. 31% on the Callhome English dataset.
paper_ai_keywords: DiarizationLM large language models (LLM) speaker diarization system diarized transcript word diarization error rate (WDER) automatic speech recognition (ASR) compact textual format prompt optionally finetuned LLM refined diarization results enhancement off-the-shelf ASR speaker diarization systems PaLM 2-S Fisher telephone conversation dataset Callhome English dataset