title: Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk
summary: Large language models (LLMs) are powerful dialogue agents, but specializing
  them towards fulfilling a specific function can be challenging. Instructing
  tuning, i.e. tuning models on instruction and sample responses generated by
  humans (Ouyang et al., 2022), has proven as an effective method to do so, yet
  requires a number of data samples that a) might not be available or b) costly
  to generate. Furthermore, this cost increases when the goal is to make the LLM
  follow a specific workflow within a dialogue instead of single instructions.
  Inspired by the self-play technique in reinforcement learning and the use of
  LLMs to simulate human agents, we propose a more effective method for data
  collection through LLMs engaging in a conversation in various roles. This
  approach generates a training data via "self-talk" of LLMs that can be refined
  and utilized for supervised fine-tuning. We introduce an automated way to
  measure the (partial) success of a dialogue. This metric is used to filter the
  generated conversational data that is fed back in LLM for training. Based on
  our automated and human evaluations of conversation quality, we demonstrate
  that such self-talk data improves results. In addition, we examine the various
  characteristics that showcase the quality of generated dialogues and how they
  can be connected to their potential utility as training data.
paper_authors._id: 659f5931210bd4a54e7fb38a
paper_authors.name: Dennis Ulmer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:59:25.771Z
paper_authors._id: 6389e86a92ad533329bf00fe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669982276443-noauth.jpeg
paper_authors.fullname: Dennis Ulmer
paper_authors.type: user
paper_authors.user: kaleidophon
paper_authors._id: 659f5931210bd4a54e7fb38b
paper_authors.name: Elman Mansimov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:59:35.919Z
paper_authors._id: 61f352fb4eba72baea21ea1e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1643336441806-noauth.png
paper_authors.fullname: Elman Mansimov
paper_authors.type: user
paper_authors.user: mansimov
paper_authors._id: 659f5931210bd4a54e7fb38c
paper_authors.name: Kaixiang Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:59:47.945Z
paper_authors._id: 64f7f197b2c85cf9e5f4a1be
paper_authors.avatarUrl: /avatars/645fd920492c16c90f86fffc0823a997.svg
paper_authors.fullname: Lin Kaixiang
paper_authors.type: user
paper_authors.user: LeoSuk
paper_authors._id: 659f5931210bd4a54e7fb38d
paper_authors.name: Justin Sun
paper_authors._id: 659f5931210bd4a54e7fb38e
paper_authors.name: Xibin Gao
paper_authors._id: 659f5931210bd4a54e7fb38f
paper_authors.name: Yi Zhang
paper_title: Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk
paper_summary: Large language models (LLMs) are powerful dialogue agents, but specializing
  them towards fulfilling a specific function can be challenging. Instructing
  tuning, i.e. tuning models on instruction and sample responses generated by
  humans (Ouyang et al., 2022), has proven as an effective method to do so, yet
  requires a number of data samples that a) might not be available or b) costly
  to generate. Furthermore, this cost increases when the goal is to make the LLM
  follow a specific workflow within a dialogue instead of single instructions.
  Inspired by the self-play technique in reinforcement learning and the use of
  LLMs to simulate human agents, we propose a more effective method for data
  collection through LLMs engaging in a conversation in various roles. This
  approach generates a training data via "self-talk" of LLMs that can be refined
  and utilized for supervised fine-tuning. We introduce an automated way to
  measure the (partial) success of a dialogue. This metric is used to filter the
  generated conversational data that is fed back in LLM for training. Based on
  our automated and human evaluations of conversation quality, we demonstrate
  that such self-talk data improves results. In addition, we examine the various
  characteristics that showcase the quality of generated dialogues and how they
  can be connected to their potential utility as training data.
paper_ai_summary: LLMs generate high-quality dialogue training data through self-simulated conversations, improving fine-tuning results and evaluated by automated metrics.
paper_ai_keywords: large language models instructing tuning self-play technique reinforcement learning supervised fine-tuning dialogue quality automated evaluation