title: Faithful Persona-based Conversational Dataset Generation with Large
    Language Models
summary: High-quality conversational datasets are essential for developing AI models
  that can communicate with users. One way to foster deeper interactions between
  a chatbot and its user is through personas, aspects of the user's character
  that provide insights into their personality, motivations, and behaviors.
  Training Natural Language Processing (NLP) models on a diverse and
  comprehensive persona-based dataset can lead to conversational models that
  create a deeper connection with the user, and maintain their engagement. In
  this paper, we leverage the power of Large Language Models (LLMs) to create a
  large, high-quality conversational dataset from a seed dataset. We propose a
  Generator-Critic architecture framework to expand the initial dataset, while
  improving the quality of its conversations. The Generator is an LLM prompted to
  output conversations. The Critic consists of a mixture of expert LLMs that
  control the quality of the generated conversations. These experts select the
  best generated conversations, which we then use to improve the Generator. We
  release Synthetic-Persona-Chat, consisting of 20k conversations seeded from
  Persona-Chat. We evaluate the quality of Synthetic-Persona-Chat and our
  generation framework on different dimensions through extensive experiments, and
  observe that the losing rate of Synthetic-Persona-Chat against Persona-Chat
  during Turing test decreases from 17.2% to 8.8% over three iterations.
paper_authors._id: 657fb590138b7e3914bdda20
paper_authors.name: Pegah Jandaghi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-12-18T07:07:37.039Z
paper_authors._id: 648cc6cb3945927fe05e2c30
paper_authors.avatarUrl: /avatars/75ace8f1e9ce516fc4543e6c11bbe6e7.svg
paper_authors.fullname: Pegah
paper_authors.type: user
paper_authors.user: JPegah
paper_authors._id: 657fb590138b7e3914bdda21
paper_authors.name: XiangHai Sheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-18T11:29:53.750Z
paper_authors._id: 6430960c2a7fbdfb5258249f
paper_authors.avatarUrl: /avatars/20f95ed65cc321f03f2c510ff1ab9b61.svg
paper_authors.fullname: Xianghai Sheng
paper_authors.type: user
paper_authors.user: nkstnkst
paper_authors._id: 657fb590138b7e3914bdda22
paper_authors.name: Xinyi Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-18T11:30:01.435Z
paper_authors._id: 6438baec27a3c24cf4a58d01
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/mRebsJ0VluWWoGhuN6jUb.jpeg
paper_authors.fullname: Xinyi Bai
paper_authors.type: user
paper_authors.user: wwwbxy123
paper_authors._id: 657fb590138b7e3914bdda23
paper_authors.name: Jay Pujara
paper_authors._id: 657fb590138b7e3914bdda24
paper_authors.name: Hakim Sidahmed
paper_title: Faithful Persona-based Conversational Dataset Generation with Large
    Language Models
paper_summary: High-quality conversational datasets are essential for developing AI models
  that can communicate with users. One way to foster deeper interactions between
  a chatbot and its user is through personas, aspects of the user's character
  that provide insights into their personality, motivations, and behaviors.
  Training Natural Language Processing (NLP) models on a diverse and
  comprehensive persona-based dataset can lead to conversational models that
  create a deeper connection with the user, and maintain their engagement. In
  this paper, we leverage the power of Large Language Models (LLMs) to create a
  large, high-quality conversational dataset from a seed dataset. We propose a
  Generator-Critic architecture framework to expand the initial dataset, while
  improving the quality of its conversations. The Generator is an LLM prompted to
  output conversations. The Critic consists of a mixture of expert LLMs that
  control the quality of the generated conversations. These experts select the
  best generated conversations, which we then use to improve the Generator. We
  release Synthetic-Persona-Chat, consisting of 20k conversations seeded from
  Persona-Chat. We evaluate the quality of Synthetic-Persona-Chat and our
  generation framework on different dimensions through extensive experiments, and
  observe that the losing rate of Synthetic-Persona-Chat against Persona-Chat
  during Turing test decreases from 17.2% to 8.8% over three iterations.
paper_ai_keywords: Large Language Models (LLMs) Generator-Critic architecture Persona-Chat Synthetic-Persona-Chat Turing test