title: NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of
    Large Language Models
summary: Large language models (LLMs) combined with tool learning have gained
  impressive results in real-world applications. During tool learning, LLMs may
  call multiple tools in nested orders, where the latter tool call may take the
  former response as its input parameters. However, current research on the
  nested tool learning capabilities is still under-explored, since the existing
  benchmarks lack of relevant data instances. To address this problem, we
  introduce NesTools to bridge the current gap in comprehensive nested tool
  learning evaluations. NesTools comprises a novel automatic data generation
  method to construct large-scale nested tool calls with different nesting
  structures. With manual review and refinement, the dataset is in high quality
  and closely aligned with real-world scenarios. Therefore, NesTools can serve as
  a new benchmark to evaluate the nested tool learning abilities of LLMs. We
  conduct extensive experiments on 22 LLMs, and provide in-depth analyses with
  NesTools, which shows that current LLMs still suffer from the complex nested
  tool learning task.
paper_authors._id: 670fcd99cda6c3cb0b2f59d6
paper_authors.name: Han Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-03T16:00:42.051Z
paper_authors._id: 644431e11bc692d87b272e71
paper_authors.avatarUrl: /avatars/798c2684c62a2fd9ac925c628abaceea.svg
paper_authors.fullname: hhan
paper_authors.type: user
paper_authors.user: codingddd
paper_authors._id: 670fcd99cda6c3cb0b2f59d7
paper_authors.name: Tong Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-17T11:26:09.076Z
paper_authors._id: 629454301ae2138079f7ff31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/629454301ae2138079f7ff31/rVtbF-j06gDiYzomTeVTc.jpeg
paper_authors.fullname: Tong Zhu
paper_authors.type: user
paper_authors.user: Spico
paper_authors._id: 670fcd99cda6c3cb0b2f59d8
paper_authors.name: Xiang Zhang
paper_authors._id: 670fcd99cda6c3cb0b2f59d9
paper_authors.name: Mengsong Wu
paper_authors._id: 670fcd99cda6c3cb0b2f59da
paper_authors.name: Hao Xiong
paper_authors._id: 670fcd99cda6c3cb0b2f59db
paper_authors.name: Wenliang Chen
paper_title: NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of
    Large Language Models
paper_summary: Large language models (LLMs) combined with tool learning have gained
  impressive results in real-world applications. During tool learning, LLMs may
  call multiple tools in nested orders, where the latter tool call may take the
  former response as its input parameters. However, current research on the
  nested tool learning capabilities is still under-explored, since the existing
  benchmarks lack of relevant data instances. To address this problem, we
  introduce NesTools to bridge the current gap in comprehensive nested tool
  learning evaluations. NesTools comprises a novel automatic data generation
  method to construct large-scale nested tool calls with different nesting
  structures. With manual review and refinement, the dataset is in high quality
  and closely aligned with real-world scenarios. Therefore, NesTools can serve as
  a new benchmark to evaluate the nested tool learning abilities of LLMs. We
  conduct extensive experiments on 22 LLMs, and provide in-depth analyses with
  NesTools, which shows that current LLMs still suffer from the complex nested
  tool learning task.
paper_ai_keywords: LLMs (Large Language Models) AG learning nested tool learning tool calls NesTools automatic data generation benchmark nested tool calls nesting structures manual review iterative compression high-quality data real-world scenarios challenging nested tool learning task