title: Source2Synth: Synthetic Data Generation and Curation Grounded in Real
    Data Sources
summary: Large Language Models still struggle in challenging scenarios that leverage
  structured data, complex reasoning, or tool usage. In this paper, we propose
  Source2Synth: a new method that can be used for teaching LLMs new skills
  without relying on costly human annotations. Source2Synth takes as input a
  custom data source and produces synthetic data points with intermediate
  reasoning steps grounded in real-world sources. Source2Synth improves the
  dataset quality by discarding low-quality generations based on their
  answerability. We demonstrate the generality of this approach by applying it to
  two challenging domains: we test reasoning abilities in multi-hop question
  answering (MHQA), and tool usage in tabular question answering (TQA). Our
  method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on
  HotPotQA compared to the fine-tuned baselines.
paper_authors._id: 66e39c75444055a570a17890
paper_authors.name: Alisia Lupidi
paper_authors._id: 66e39c75444055a570a17891
paper_authors.name: Carlos Gemmell
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:01:56.124Z
paper_authors._id: 654ce9abce215f3cd11f8df7
paper_authors.avatarUrl: /avatars/4139d3956cf8b1bb2ebd820e9b031123.svg
paper_authors.fullname: Carlos Gemmell
paper_authors.type: user
paper_authors.user: carlos-gemmell-malt-ai
paper_authors._id: 66e39c75444055a570a17892
paper_authors.name: Nicola Cancedda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:02:02.257Z
paper_authors._id: 65d8ee3a2b0037ec192f7c20
paper_authors.avatarUrl: /avatars/900fe214eb32279b5dfb1a8ff0ef4be1.svg
paper_authors.fullname: Nicola Cancedda
paper_authors.type: user
paper_authors.user: ncan
paper_authors._id: 66e39c75444055a570a17893
paper_authors.name: Jane Dwivedi-Yu
paper_authors._id: 66e39c75444055a570a17894
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:02:17.791Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 66e39c75444055a570a17895
paper_authors.name: Jakob Foerster
paper_authors._id: 66e39c75444055a570a17896
paper_authors.name: Roberta Raileanu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:02:41.162Z
paper_authors._id: 633e94793a17ab61de8e2b9c
paper_authors.avatarUrl: /avatars/5f2f58ddeed211393660ada6b135f0d5.svg
paper_authors.fullname: Roberta Raileanu
paper_authors.type: user
paper_authors.user: rraileanu
paper_authors._id: 66e39c75444055a570a17897
paper_authors.name: Maria Lomeli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:02:54.467Z
paper_authors._id: 64f86587e7584abc62545b3e
paper_authors.avatarUrl: /avatars/57296279592ad5f77c39d174f6289420.svg
paper_authors.fullname: Maria Lomeli
paper_authors.type: user
paper_authors.user: marialomeli
paper_title: Source2Synth: Synthetic Data Generation and Curation Grounded in Real
    Data Sources
paper_summary: Large Language Models still struggle in challenging scenarios that leverage
  structured data, complex reasoning, or tool usage. In this paper, we propose
  Source2Synth: a new method that can be used for teaching LLMs new skills
  without relying on costly human annotations. Source2Synth takes as input a
  custom data source and produces synthetic data points with intermediate
  reasoning steps grounded in real-world sources. Source2Synth improves the
  dataset quality by discarding low-quality generations based on their
  answerability. We demonstrate the generality of this approach by applying it to
  two challenging domains: we test reasoning abilities in multi-hop question
  answering (MHQA), and tool usage in tabular question answering (TQA). Our
  method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on
  HotPotQA compared to the fine-tuned baselines.
paper_ai_summary: Source2Synth improves LLM performance in structured reasoning and tool usage scenarios by generating high-quality synthetic data points and filtering out low-quality ones.
paper_ai_keywords: Large Language Models multi-hop question answering (MHQA) tabular question answering (TQA) synthetic data answerability filtering Source2Synth WikiSQL HotPotQA