Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
<|reference_start|>Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources: Large Language Models still struggle in challenging scenarios that leverage structured data, complex reasoning, or tool usage. In this paper, we propose Source2Synth: a new method that can be used for teaching LLMs new skills without relying on costly human annotations. Source2Synth takes as input a custom data source and produces synthetic data points with intermediate reasoning steps grounded in real-world sources. Source2Synth improves the dataset quality by discarding low-quality generations based on their answerability. We demonstrate the generality of this approach by applying it to two challenging domains: we test reasoning abilities in multi-hop question answering (MHQA), and tool usage in tabular question answering (TQA). Our method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on HotPotQA compared to the fine-tuned baselines.<|reference_end|>
@article{lupidi2024source2synth:,
    title={Source2Synth: Synthetic Data Generation and Curation Grounded in Real
  Data Sources},
    author={Alisia Lupidi, Carlos Gemmell, Nicola Cancedda, Jane Dwivedi-Yu, Jason
  Weston, Jakob Foerster, Roberta Raileanu, Maria Lomeli},
    journal={arXiv preprint arXiv:2409.08239},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.08239},
    primaryClass={cs.CL cs.AI}
}