APIGen: Automated Pipeline for Generating Verifiable and Diverse
  Function-Calling Datasets
The advancement of function-calling agent models requires diverse, reliable,
and high-quality datasets. This paper presents APIGen, an automated data
generation pipeline designed to synthesize verifiable high-quality datasets for
function-calling applications. We leverage APIGen and collect 3,673 executable
APIs across 21 different categories to generate diverse function-calling
datasets in a scalable and structured manner. Each data in our dataset is
verified through three hierarchical stages: format checking, actual function
executions, and semantic verification, ensuring its reliability and
correctness. We demonstrate that models trained with our curated datasets, even
with only 7B parameters, can achieve state-of-the-art performance on the
Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models.
Moreover, our 1B model achieves exceptional performance, surpassing
GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000
high-quality entries, aiming to advance the field of function-calling agent
domains. The dataset is available on Huggingface:
https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the
project homepage: https://apigen-pipeline.github.io/
667cba7064aafdeb4795fc2e
Zuxin Liu
claimed_verified
2024-06-30T20:57:42.174Z
638e971f2cc490759fe9210e
/avatars/9578d0badc084d340c028a59e571abd3.svg
Liu
user
Zuxin
667cba7064aafdeb4795fc2f
Thai Hoang
667cba7064aafdeb4795fc30
Jianguo Zhang
claimed_verified
2024-06-27T07:31:35.196Z
630fcbd758d83e8f64d82777
https://cdn-avatars.huggingface.co/v1/production/uploads/630fcbd758d83e8f64d82777/0jwTa1MzeXZbDtJ-uXGo6.jpeg
Jianguo Zhang
user
jianguozhang
667cba7064aafdeb4795fc31
Ming Zhu
claimed_verified
2024-06-27T18:48:07.997Z
624376052f7aa3b4ec5b3acc
/avatars/b2a66e683ec183f346d4508acedc9bc5.svg
Ming Zhu
user
mingzhu0527
667cba7064aafdeb4795fc32
Tian Lan
667cba7064aafdeb4795fc33
Shirley Kokane
claimed_verified
2025-01-08T09:48:40.719Z
65e0e9c8b898f6b6a1847095
/avatars/bbab6a61b912bf287bac76ef72e2f99a.svg
Shirley Kokane
user
skokane
667cba7064aafdeb4795fc34
Juntao Tan
667cba7064aafdeb4795fc35
Weiran Yao
claimed_verified
2024-06-27T07:31:37.366Z
632cdea254e2c512c8f95b12
/avatars/a6d06cdd75861ae7d589f1343d81a5c5.svg
Weiran Yao
user
weirayao
667cba7064aafdeb4795fc36
Zhiwei Liu
667cba7064aafdeb4795fc37
Yihao Feng
667cba7064aafdeb4795fc38
Rithesh Murthy
claimed_verified
2024-06-28T08:03:12.079Z
60381555c58b41eeda0d6ccf
/avatars/2c2c4b75dd0199478dc0fbc72f9bcadc.svg
rmurthy
user
rmurthy
667cba7064aafdeb4795fc39
Liangwei Yang
667cba7064aafdeb4795fc3a
Silvio Savarese
667cba7064aafdeb4795fc3b
Juan Carlos Niebles
claimed_verified
2024-08-19T22:14:04.702Z
65bb26b5d61b51a50805aba2
/avatars/966c8469f55776948a7cf42d98accff3.svg
Juan Carlos Niebles
user
niebles
667cba7064aafdeb4795fc3c
Huan Wang
667cba7064aafdeb4795fc3d
Shelby Heinecke
667cba7064aafdeb4795fc3e
Caiming Xiong
APIGen: Automated Pipeline for Generating Verifiable and Diverse
  Function-Calling Datasets
The advancement of function-calling agent models requires diverse, reliable,
and high-quality datasets. This paper presents APIGen, an automated data
generation pipeline designed to synthesize verifiable high-quality datasets for
function-calling applications. We leverage APIGen and collect 3,673 executable
APIs across 21 different categories to generate diverse function-calling
datasets in a scalable and structured manner. Each data in our dataset is
verified through three hierarchical stages: format checking, actual function
executions, and semantic verification, ensuring its reliability and
correctness. We demonstrate that models trained with our curated datasets, even
with only 7B parameters, can achieve state-of-the-art performance on the
Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models.
Moreover, our 1B model achieves exceptional performance, surpassing
GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000
high-quality entries, aiming to advance the field of function-calling agent
domains. The dataset is available on Huggingface:
https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the
project homepage: https://apigen-pipeline.github.io/
APIGen is an automated pipeline for generating high-quality, verified datasets for function-calling applications, demonstrating superior performance in the Berkeley Function-Calling Benchmark.
function-calling agent models APIGen automated data generation pipeline executable APIs dataset verification format checking semantic verification Berkeley Function-Calling Benchmark GPT-4 GPT-3.5-Turbo Claude-3 Haiku