APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
<|reference_start|>APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets: The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/<|reference_end|>
@article{liu2024apigen:,
    title={APIGen: Automated Pipeline for Generating Verifiable and Diverse
  Function-Calling Datasets},
    author={Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley
  Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy,
  Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby
  Heinecke, Caiming Xiong},
    journal={arXiv preprint arXiv:2406.18518},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.18518},
    primaryClass={cs.CL cs.AI cs.LG cs.SE}
}