title: Augmenting text for spoken language understanding with Large Language
    Models
summary: Spoken semantic parsing (SSP) involves generating machine-comprehensible
  parses from input speech. Training robust models for existing application
  domains represented in training data or extending to new domains requires
  corresponding triplets of speech-transcript-semantic parse data, which is
  expensive to obtain. In this paper, we address this challenge by examining
  methods that can use transcript-semantic parse data (unpaired text) without
  corresponding speech. First, when unpaired text is drawn from existing textual
  corpora, Joint Audio Text (JAT) and Text-to-Speech (TTS) are compared as ways
  to generate speech representations for unpaired text. Experiments on the STOP
  dataset show that unpaired text from existing and new domains improves
  performance by 2% and 30% in absolute Exact Match (EM) respectively. Second, we
  consider the setting when unpaired text is not available in existing textual
  corpora. We propose to prompt Large Language Models (LLMs) to generate unpaired
  text for existing and new domains. Experiments show that examples and words
  that co-occur with intents can be used to generate unpaired text with Llama
  2.0. Using the generated text with JAT and TTS for spoken semantic parsing
  improves EM on STOP by 1.4% and 2.6% absolute for existing and new domains
  respectively.
paper_authors._id: 65090eda948ce5dce89bb87f
paper_authors.name: Roshan Sharma
paper_authors._id: 65090eda948ce5dce89bb880
paper_authors.name: Suyoun Kim
paper_authors._id: 65090eda948ce5dce89bb881
paper_authors.name: Daniel Lazar
paper_authors._id: 65090eda948ce5dce89bb882
paper_authors.name: Trang Le
paper_authors._id: 65090eda948ce5dce89bb883
paper_authors.name: Akshat Shrivastava
paper_authors._id: 65090eda948ce5dce89bb884
paper_authors.name: Kwanghoon Ahn
paper_authors._id: 65090eda948ce5dce89bb885
paper_authors.name: Piyush Kansal
paper_authors._id: 65090eda948ce5dce89bb886
paper_authors.name: Leda Sari
paper_authors._id: 65090eda948ce5dce89bb887
paper_authors.name: Ozlem Kalinli
paper_authors._id: 65090eda948ce5dce89bb888
paper_authors.name: Michael Seltzer
paper_title: Augmenting text for spoken language understanding with Large Language
    Models
paper_summary: Spoken semantic parsing (SSP) involves generating machine-comprehensible
  parses from input speech. Training robust models for existing application
  domains represented in training data or extending to new domains requires
  corresponding triplets of speech-transcript-semantic parse data, which is
  expensive to obtain. In this paper, we address this challenge by examining
  methods that can use transcript-semantic parse data (unpaired text) without
  corresponding speech. First, when unpaired text is drawn from existing textual
  corpora, Joint Audio Text (JAT) and Text-to-Speech (TTS) are compared as ways
  to generate speech representations for unpaired text. Experiments on the STOP
  dataset show that unpaired text from existing and new domains improves
  performance by 2% and 30% in absolute Exact Match (EM) respectively. Second, we
  consider the setting when unpaired text is not available in existing textual
  corpora. We propose to prompt Large Language Models (LLMs) to generate unpaired
  text for existing and new domains. Experiments show that examples and words
  that co-occur with intents can be used to generate unpaired text with Llama
  2.0. Using the generated text with JAT and TTS for spoken semantic parsing
  improves EM on STOP by 1.4% and 2.6% absolute for existing and new domains
  respectively.
paper_ai_summary: Methods using transcript-semantic parse data without corresponding speech improve spoken semantic parsing performance, especially for new domains, through Joint Audio Text and Text-to-Speech techniques, and by prompting Large Language Models to generate unpaired text.
paper_ai_keywords: Joint Audio Text (JAT) Text-to-Speech (TTS) Large Language Models (LLMs) spoken semantic parsing transcript-semantic parse data Exact Match (EM) STOP dataset unpaired text