title: API-BLEND: A Comprehensive Corpora for Training and Benchmarking API
    LLMs
summary: There is a growing need for Large Language Models (LLMs) to effectively use
  tools and external Application Programming Interfaces (APIs) to plan and
  complete tasks. As such, there is tremendous interest in methods that can
  acquire sufficient quantities of train and test data that involve calls to
  tools / APIs. Two lines of research have emerged as the predominant strategies
  for addressing this challenge. The first has focused on synthetic data
  generation techniques, while the second has involved curating task-adjacent
  datasets which can be transformed into API / Tool-based tasks. In this paper,
  we focus on the task of identifying, curating, and transforming existing
  datasets and, in turn, introduce API-BLEND, a large corpora for training and
  systematic testing of tool-augmented LLMs. The datasets mimic real-world
  scenarios involving API-tasks such as API / tool detection, slot filling, and
  sequencing of the detected APIs. We demonstrate the utility of the API-BLEND
  dataset for both training and benchmarking purposes.
paper_authors._id: 65dc073f0af7e21ba4f8c5eb
paper_authors.name: Kinjal Basu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-26T21:09:40.779Z
paper_authors._id: 64a6c818e3a28de0a4ccc23a
paper_authors.avatarUrl: /avatars/0aeca86e12aebbfff7ad1a28691c04ec.svg
paper_authors.fullname: Kinjal Basu
paper_authors.type: user
paper_authors.user: kinjalbasu
paper_authors._id: 65dc073f0af7e21ba4f8c5ec
paper_authors.name: Ibrahim Abdelaziz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-07T08:33:58.229Z
paper_authors._id: 6632b81cab69b961af2fcfb4
paper_authors.avatarUrl: /avatars/05b0af95a30b3426e4408f658293ec43.svg
paper_authors.fullname: Ibrahim A
paper_authors.type: user
paper_authors.user: ibrahimabdelaziz
paper_authors._id: 65dc073f0af7e21ba4f8c5ed
paper_authors.name: Subhajit Chaudhury
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-26T21:09:38.105Z
paper_authors._id: 63536559a023833e59da3458
paper_authors.avatarUrl: /avatars/bcf1c4193f75bb464998a02403f1eb91.svg
paper_authors.fullname: Subhajit Chaudhury
paper_authors.type: user
paper_authors.user: cosmik1412
paper_authors._id: 65dc073f0af7e21ba4f8c5ee
paper_authors.name: Soham Dan
paper_authors._id: 65dc073f0af7e21ba4f8c5ef
paper_authors.name: Maxwell Crouse
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-28T08:28:24.702Z
paper_authors._id: 65de20ae5522923cf719a744
paper_authors.avatarUrl: /avatars/087eee3d917c882830998a97baf3ba86.svg
paper_authors.fullname: Maxwell Crouse
paper_authors.type: user
paper_authors.user: mvcrouse
paper_authors._id: 65dc073f0af7e21ba4f8c5f0
paper_authors.name: Asim Munawar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-26T14:34:23.811Z
paper_authors._id: 647959323b7f8b1f624c9666
paper_authors.avatarUrl: /avatars/f6cad73bcfaa26c6cc4d01727d6a2ba1.svg
paper_authors.fullname: Asim Munawar
paper_authors.type: user
paper_authors.user: asimmunawar
paper_authors._id: 65dc073f0af7e21ba4f8c5f1
paper_authors.name: Sadhana Kumaravel
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-06-10T17:47:09.321Z
paper_authors._id: 66673a677a87ddaf0f99f002
paper_authors.avatarUrl: /avatars/fd26ca86c9a5853a4cd35f68b0e5935d.svg
paper_authors.fullname: Sadhana Kumaravel
paper_authors.type: user
paper_authors.user: sadhanak
paper_authors._id: 65dc073f0af7e21ba4f8c5f2
paper_authors.name: Vinod Muthusamy
paper_authors._id: 65dc073f0af7e21ba4f8c5f3
paper_authors.name: Pavan Kapanipathi
paper_authors._id: 65dc073f0af7e21ba4f8c5f4
paper_authors.name: Luis A. Lastras
paper_title: API-BLEND: A Comprehensive Corpora for Training and Benchmarking API
    LLMs
paper_summary: There is a growing need for Large Language Models (LLMs) to effectively use
  tools and external Application Programming Interfaces (APIs) to plan and
  complete tasks. As such, there is tremendous interest in methods that can
  acquire sufficient quantities of train and test data that involve calls to
  tools / APIs. Two lines of research have emerged as the predominant strategies
  for addressing this challenge. The first has focused on synthetic data
  generation techniques, while the second has involved curating task-adjacent
  datasets which can be transformed into API / Tool-based tasks. In this paper,
  we focus on the task of identifying, curating, and transforming existing
  datasets and, in turn, introduce API-BLEND, a large corpora for training and
  systematic testing of tool-augmented LLMs. The datasets mimic real-world
  scenarios involving API-tasks such as API / tool detection, slot filling, and
  sequencing of the detected APIs. We demonstrate the utility of the API-BLEND
  dataset for both training and benchmarking purposes.
paper_ai_keywords: Large Language Models (LLMs) tools Application Programming Interfaces (APIs) synthetic data generation task-adjacent datasets API-BLEND API-tasks API / tool detection slot filling sequencing