title: Genie: Achieving Human Parity in Content-Grounded Datasets Generation
summary: The lack of high-quality data for content-grounded generation tasks has been
  identified as a major obstacle to advancing these tasks. To address this gap,
  we propose Genie, a novel method for automatically generating high-quality
  content-grounded data. It consists of three stages: (a) Content Preparation,
  (b) Generation: creating task-specific examples from the content (e.g.,
  question-answer pairs or summaries). (c) Filtering mechanism aiming to ensure
  the quality and faithfulness of the generated data. We showcase this
  methodology by generating three large-scale synthetic data, making wishes, for
  Long-Form Question-Answering (LFQA), summarization, and information extraction.
  In a human evaluation, our generated data was found to be natural and of high
  quality. Furthermore, we compare models trained on our data with models trained
  on human-written data -- ELI5 and ASQA for LFQA and CNN-DailyMail for
  Summarization. We show that our models are on par with or outperforming models
  trained on human-generated data and consistently outperforming them in
  faithfulness. Finally, we applied our method to create LFQA data within the
  medical domain and compared a model trained on it with models trained on other
  domains.
paper_authors._id: 65b32fb506d8b55123d4c0f7
paper_authors.name: Asaf Yehudai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:16:35.021Z
paper_authors._id: 638324f862badff43269e588
paper_authors.avatarUrl: /avatars/907a39a9b44fc8b7f3fad35858b01fb7.svg
paper_authors.fullname: Asaf Yehudai
paper_authors.type: user
paper_authors.user: Asaf-Yehudai
paper_authors._id: 65b32fb506d8b55123d4c0f8
paper_authors.name: Boaz Carmeli
paper_authors._id: 65b32fb506d8b55123d4c0f9
paper_authors.name: Yosi Mass
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:16:46.254Z
paper_authors._id: 62efa2c5d88b150b068e7c32
paper_authors.avatarUrl: /avatars/0e99bb420dbb34a10cdb56c00cdc2a50.svg
paper_authors.fullname: yosi mass
paper_authors.type: user
paper_authors.user: yosimass
paper_authors._id: 65b32fb506d8b55123d4c0fa
paper_authors.name: Ofir Arviv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:16:52.056Z
paper_authors._id: 632ae2606cb20ba0ae8007d8
paper_authors.avatarUrl: /avatars/023eb4536e9bfcebbcc42eaea7167596.svg
paper_authors.fullname: Ofir Arviv
paper_authors.type: user
paper_authors.user: OfirArviv
paper_authors._id: 65b32fb506d8b55123d4c0fb
paper_authors.name: Nathaniel Mills
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:16:59.730Z
paper_authors._id: 60106cc0dee0d06a3a436bc4
paper_authors.avatarUrl: /avatars/61961ba0215110083ed539410e230719.svg
paper_authors.fullname: Nathaniel Mills
paper_authors.type: user
paper_authors.user: wnmills3
paper_authors._id: 65b32fb506d8b55123d4c0fc
paper_authors.name: Assaf Toledo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:17:06.218Z
paper_authors._id: 62931fc15d1c94a62e7e66df
paper_authors.avatarUrl: /avatars/3312f9fa92b04c09203a4820d2abb70f.svg
paper_authors.fullname: Assaf Toledo
paper_authors.type: user
paper_authors.user: assaft
paper_authors._id: 65b32fb506d8b55123d4c0fd
paper_authors.name: Eyal Shnarch
paper_authors._id: 65b32fb506d8b55123d4c0fe
paper_authors.name: Leshem Choshen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T10:17:15.398Z
paper_authors._id: 61bf40824b4300d0fb0acf59
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644224872623-61bf40824b4300d0fb0acf59.jpeg
paper_authors.fullname: Leshem Choshen
paper_authors.type: user
paper_authors.user: borgr
paper_title: Genie: Achieving Human Parity in Content-Grounded Datasets Generation
paper_summary: The lack of high-quality data for content-grounded generation tasks has been
  identified as a major obstacle to advancing these tasks. To address this gap,
  we propose Genie, a novel method for automatically generating high-quality
  content-grounded data. It consists of three stages: (a) Content Preparation,
  (b) Generation: creating task-specific examples from the content (e.g.,
  question-answer pairs or summaries). (c) Filtering mechanism aiming to ensure
  the quality and faithfulness of the generated data. We showcase this
  methodology by generating three large-scale synthetic data, making wishes, for
  Long-Form Question-Answering (LFQA), summarization, and information extraction.
  In a human evaluation, our generated data was found to be natural and of high
  quality. Furthermore, we compare models trained on our data with models trained
  on human-written data -- ELI5 and ASQA for LFQA and CNN-DailyMail for
  Summarization. We show that our models are on par with or outperforming models
  trained on human-generated data and consistently outperforming them in
  faithfulness. Finally, we applied our method to create LFQA data within the
  medical domain and compared a model trained on it with models trained on other
  domains.
paper_ai_summary: A method called Genie automatically generates high-quality content-grounded data for specific tasks, outperforming models trained on human-generated data in faithfulness and quality.
paper_ai_keywords: Content Preparation Generation Filtering mechanism Long-Form Question-Answering (LFQA) summarization information extraction synthetic data human evaluation faithfulness