NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of
  Large Language Models
Large language models (LLMs) combined with tool learning have gained
impressive results in real-world applications. During tool learning, LLMs may
call multiple tools in nested orders, where the latter tool call may take the
former response as its input parameters. However, current research on the
nested tool learning capabilities is still under-explored, since the existing
benchmarks lack of relevant data instances. To address this problem, we
introduce NesTools to bridge the current gap in comprehensive nested tool
learning evaluations. NesTools comprises a novel automatic data generation
method to construct large-scale nested tool calls with different nesting
structures. With manual review and refinement, the dataset is in high quality
and closely aligned with real-world scenarios. Therefore, NesTools can serve as
a new benchmark to evaluate the nested tool learning abilities of LLMs. We
conduct extensive experiments on 22 LLMs, and provide in-depth analyses with
NesTools, which shows that current LLMs still suffer from the complex nested
tool learning task.
670fcd99cda6c3cb0b2f59d6
Han Han
claimed_verified
2025-02-03T16:00:42.051Z
644431e11bc692d87b272e71
/avatars/798c2684c62a2fd9ac925c628abaceea.svg
hhan
user
codingddd
670fcd99cda6c3cb0b2f59d7
Tong Zhu
claimed_verified
2024-10-17T11:26:09.076Z
629454301ae2138079f7ff31
https://cdn-avatars.huggingface.co/v1/production/uploads/629454301ae2138079f7ff31/rVtbF-j06gDiYzomTeVTc.jpeg
Tong Zhu
user
Spico
670fcd99cda6c3cb0b2f59d8
Xiang Zhang
670fcd99cda6c3cb0b2f59d9
Mengsong Wu
claimed_verified
2025-03-25T08:22:50.233Z
62b31667cbd2a402fc5e5b0f
https://cdn-avatars.huggingface.co/v1/production/uploads/1655903835218-noauth.jpeg
wums
user
RadiCat
670fcd99cda6c3cb0b2f59da
Hao Xiong
670fcd99cda6c3cb0b2f59db
Wenliang Chen
NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of
  Large Language Models
Large language models (LLMs) combined with tool learning have gained
impressive results in real-world applications. During tool learning, LLMs may
call multiple tools in nested orders, where the latter tool call may take the
former response as its input parameters. However, current research on the
nested tool learning capabilities is still under-explored, since the existing
benchmarks lack of relevant data instances. To address this problem, we
introduce NesTools to bridge the current gap in comprehensive nested tool
learning evaluations. NesTools comprises a novel automatic data generation
method to construct large-scale nested tool calls with different nesting
structures. With manual review and refinement, the dataset is in high quality
and closely aligned with real-world scenarios. Therefore, NesTools can serve as
a new benchmark to evaluate the nested tool learning abilities of LLMs. We
conduct extensive experiments on 22 LLMs, and provide in-depth analyses with
NesTools, which shows that current LLMs still suffer from the complex nested
tool learning task.
NesTools introduces a benchmark dataset for evaluating the nested tool learning capabilities of large language models (LLMs).
large language models (LLMs) tool learning nested tool learning NesTools automatic data generation nested tool calls benchmark dataset extensive experiments in-depth analyses