title: Toward Interactive Dictation
summary: Voice dictation is an increasingly important text input modality. Existing
  systems that allow both dictation and editing-by-voice restrict their command
  language to flat templates invoked by trigger words. In this work, we study the
  feasibility of allowing users to interrupt their dictation with spoken editing
  commands in open-ended natural language. We introduce a new task and dataset,
  TERTiUS, to experiment with such systems. To support this flexibility in
  real-time, a system must incrementally segment and classify spans of speech as
  either dictation or command, and interpret the spans that are commands. We
  experiment with using large pre-trained language models to predict the edited
  text, or alternatively, to predict a small text-editing program. Experiments
  show a natural trade-off between model accuracy and latency: a smaller model
  achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger
  model achieves 55% end-state accuracy with 7 seconds of latency.
paper_authors._id: 64acb1286864362a7efb463f
paper_authors.name: Belinda Z. Li
paper_authors._id: 64acb1286864362a7efb4640
paper_authors.name: Jason Eisner
paper_authors._id: 64acb1286864362a7efb4641
paper_authors.name: Adam Pauls
paper_authors._id: 64acb1286864362a7efb4642
paper_authors.name: Sam Thomson
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-13T08:32:59.130Z
paper_authors._id: 64aed77821d18d09271708b7
paper_authors.avatarUrl: /avatars/1c014f0c48fc85edb8e8e28a1e8bb8ae.svg
paper_authors.fullname: Samuel M Thomson
paper_authors.type: user
paper_authors.user: sammthomson
paper_title: Toward Interactive Dictation
paper_summary: Voice dictation is an increasingly important text input modality. Existing
  systems that allow both dictation and editing-by-voice restrict their command
  language to flat templates invoked by trigger words. In this work, we study the
  feasibility of allowing users to interrupt their dictation with spoken editing
  commands in open-ended natural language. We introduce a new task and dataset,
  TERTiUS, to experiment with such systems. To support this flexibility in
  real-time, a system must incrementally segment and classify spans of speech as
  either dictation or command, and interpret the spans that are commands. We
  experiment with using large pre-trained language models to predict the edited
  text, or alternatively, to predict a small text-editing program. Experiments
  show a natural trade-off between model accuracy and latency: a smaller model
  achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger
  model achieves 55% end-state accuracy with 7 seconds of latency.
paper_ai_summary: A study explores real-time spoken interruption for dictation and editing using large pre-trained language models, showing a trade-off between accuracy and latency.
paper_ai_keywords: large pre-trained language models end-state accuracy latency real-time spoken interruption dictation editing commands natural language