title: APIGen: Automated Pipeline for Generating Verifiable and Diverse
    Function-Calling Datasets
summary: The advancement of function-calling agent models requires diverse, reliable,
  and high-quality datasets. This paper presents APIGen, an automated data
  generation pipeline designed to synthesize verifiable high-quality datasets for
  function-calling applications. We leverage APIGen and collect 3,673 executable
  APIs across 21 different categories to generate diverse function-calling
  datasets in a scalable and structured manner. Each data in our dataset is
  verified through three hierarchical stages: format checking, actual function
  executions, and semantic verification, ensuring its reliability and
  correctness. We demonstrate that models trained with our curated datasets, even
  with only 7B parameters, can achieve state-of-the-art performance on the
  Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models.
  Moreover, our 1B model achieves exceptional performance, surpassing
  GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000
  high-quality entries, aiming to advance the field of function-calling agent
  domains. The dataset is available on Huggingface:
  https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the
  project homepage: https://apigen-pipeline.github.io/
paper_authors._id: 667cba7064aafdeb4795fc2e
paper_authors.name: Zuxin Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-30T20:57:42.174Z
paper_authors._id: 638e971f2cc490759fe9210e
paper_authors.avatarUrl: /avatars/9578d0badc084d340c028a59e571abd3.svg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Zuxin
paper_authors._id: 667cba7064aafdeb4795fc2f
paper_authors.name: Thai Hoang
paper_authors._id: 667cba7064aafdeb4795fc30
paper_authors.name: Jianguo Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T07:31:35.196Z
paper_authors._id: 630fcbd758d83e8f64d82777
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630fcbd758d83e8f64d82777/0jwTa1MzeXZbDtJ-uXGo6.jpeg
paper_authors.fullname: Jianguo Zhang
paper_authors.type: user
paper_authors.user: jianguozhang
paper_authors._id: 667cba7064aafdeb4795fc31
paper_authors.name: Ming Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T18:48:07.997Z
paper_authors._id: 624376052f7aa3b4ec5b3acc
paper_authors.avatarUrl: /avatars/b2a66e683ec183f346d4508acedc9bc5.svg
paper_authors.fullname: Ming Zhu
paper_authors.type: user
paper_authors.user: mingzhu0527
paper_authors._id: 667cba7064aafdeb4795fc32
paper_authors.name: Tian Lan
paper_authors._id: 667cba7064aafdeb4795fc33
paper_authors.name: Shirley Kokane
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-08T09:48:40.719Z
paper_authors._id: 65e0e9c8b898f6b6a1847095
paper_authors.avatarUrl: /avatars/bbab6a61b912bf287bac76ef72e2f99a.svg
paper_authors.fullname: Shirley Kokane
paper_authors.type: user
paper_authors.user: skokane
paper_authors._id: 667cba7064aafdeb4795fc34
paper_authors.name: Juntao Tan
paper_authors._id: 667cba7064aafdeb4795fc35
paper_authors.name: Weiran Yao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T07:31:37.366Z
paper_authors._id: 632cdea254e2c512c8f95b12
paper_authors.avatarUrl: /avatars/a6d06cdd75861ae7d589f1343d81a5c5.svg
paper_authors.fullname: Weiran Yao
paper_authors.type: user
paper_authors.user: weirayao
paper_authors._id: 667cba7064aafdeb4795fc36
paper_authors.name: Zhiwei Liu
paper_authors._id: 667cba7064aafdeb4795fc37
paper_authors.name: Yihao Feng
paper_authors._id: 667cba7064aafdeb4795fc38
paper_authors.name: Rithesh Murthy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-28T08:03:12.079Z
paper_authors._id: 60381555c58b41eeda0d6ccf
paper_authors.avatarUrl: /avatars/2c2c4b75dd0199478dc0fbc72f9bcadc.svg
paper_authors.fullname: rmurthy
paper_authors.type: user
paper_authors.user: rmurthy
paper_authors._id: 667cba7064aafdeb4795fc39
paper_authors.name: Liangwei Yang
paper_authors._id: 667cba7064aafdeb4795fc3a
paper_authors.name: Silvio Savarese
paper_authors._id: 667cba7064aafdeb4795fc3b
paper_authors.name: Juan Carlos Niebles
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-19T22:14:04.702Z
paper_authors._id: 65bb26b5d61b51a50805aba2
paper_authors.avatarUrl: /avatars/966c8469f55776948a7cf42d98accff3.svg
paper_authors.fullname: Juan Carlos Niebles
paper_authors.type: user
paper_authors.user: niebles
paper_authors._id: 667cba7064aafdeb4795fc3c
paper_authors.name: Huan Wang
paper_authors._id: 667cba7064aafdeb4795fc3d
paper_authors.name: Shelby Heinecke
paper_authors._id: 667cba7064aafdeb4795fc3e
paper_authors.name: Caiming Xiong
paper_title: APIGen: Automated Pipeline for Generating Verifiable and Diverse
    Function-Calling Datasets
paper_summary: The advancement of function-calling agent models requires diverse, reliable,
  and high-quality datasets. This paper presents APIGen, an automated data
  generation pipeline designed to synthesize verifiable high-quality datasets for
  function-calling applications. We leverage APIGen and collect 3,673 executable
  APIs across 21 different categories to generate diverse function-calling
  datasets in a scalable and structured manner. Each data in our dataset is
  verified through three hierarchical stages: format checking, actual function
  executions, and semantic verification, ensuring its reliability and
  correctness. We demonstrate that models trained with our curated datasets, even
  with only 7B parameters, can achieve state-of-the-art performance on the
  Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models.
  Moreover, our 1B model achieves exceptional performance, surpassing
  GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000
  high-quality entries, aiming to advance the field of function-calling agent
  domains. The dataset is available on Huggingface:
  https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the
  project homepage: https://apigen-pipeline.github.io/
paper_ai_summary: APIGen is an automated pipeline for generating high-quality, verified datasets for function-calling applications, demonstrating superior performance in the Berkeley Function-Calling Benchmark.
paper_ai_keywords: function-calling agent models APIGen automated data generation pipeline executable APIs dataset verification format checking semantic verification Berkeley Function-Calling Benchmark GPT-4 GPT-3.5-Turbo Claude-3 Haiku