TPTU-v2: Boosting Task Planning and Tool Usage of Large Language
  Model-based Agents in Real-world Systems
Large Language Models (LLMs) have demonstrated proficiency in addressing
tasks that necessitate a combination of task planning and the usage of external
tools that require a blend of task planning and the utilization of external
tools, such as APIs. However, real-world complex systems present three
prevalent challenges concerning task planning and tool usage: (1) The real
system usually has a vast array of APIs, so it is impossible to feed the
descriptions of all APIs to the prompt of LLMs as the token length is limited;
(2) the real system is designed for handling complex tasks, and the base LLMs
can hardly plan a correct sub-task order and API-calling order for such tasks;
(3) Similar semantics and functionalities among APIs in real systems create
challenges for both LLMs and even humans in distinguishing between them. In
response, this paper introduces a comprehensive framework aimed at enhancing
the Task Planning and Tool Usage (TPTU) abilities of LLM-based agents operating
within real-world systems. Our framework comprises three key components
designed to address these challenges: (1) the API Retriever selects the most
pertinent APIs for the user task among the extensive array available; (2) LLM
Finetuner tunes a base LLM so that the finetuned LLM can be more capable for
task planning and API calling; (3) the Demo Selector adaptively retrieves
different demonstrations related to hard-to-distinguish APIs, which is further
used for in-context learning to boost the final performance. We validate our
methods using a real-world commercial system as well as an open-sourced
academic dataset, and the outcomes clearly showcase the efficacy of each
individual component as well as the integrated framework.
655c2b6971ed242b426b2d4a
Yilun Kong
655c2b6971ed242b426b2d4b
Jingqing Ruan
admin_assigned
2023-11-21T14:08:26.534Z
64872abb3fb9bc8f4994e014
/avatars/9743b5a2b6f9b9990aa1cb06bd6eb5c6.svg
Jingqing Ruan
user
Amanda2023
655c2b6971ed242b426b2d4c
Yihong Chen
admin_assigned
2023-11-21T14:08:43.117Z
63e09700f0c75dfb876e8570
/avatars/53c17418163004a6e2730987b1cfc3b8.svg
Chenyihong
user
Ffftttlllhappy
655c2b6971ed242b426b2d4d
Bin Zhang
655c2b6971ed242b426b2d4e
Tianpeng Bao
655c2b6971ed242b426b2d4f
Shiwei Shi
655c2b6971ed242b426b2d50
Guoqing Du
655c2b6971ed242b426b2d51
Xiaoru Hu
extracted_pending
2024-12-16T07:43:34.434Z
670f7471fcbfebab8561d343
/avatars/79ea08c9d4c359afdf935da107e0b64f.svg
XR
user
HU0016
655c2b6971ed242b426b2d52
Hangyu Mao
extracted_confirmed
2023-11-22T02:13:27.732Z
655d62c5ea26da0aca92746d
/avatars/74144ab0d0c5c23769cfc2f33ad87a7a.svg
Hangyu Mao
user
HangyuMao
655c2b6971ed242b426b2d53
Ziyue Li
655c2b6971ed242b426b2d54
Xingyu Zeng
655c2b6971ed242b426b2d55
Rui Zhao
extracted_confirmed
2025-06-08T11:28:42.666Z
6405d7cca577649430bcceb9
/avatars/befc09d3bc82685c8e205dcadfec13c0.svg
Rui Zhao
user
Robert0812
TPTU-v2: Boosting Task Planning and Tool Usage of Large Language
  Model-based Agents in Real-world Systems
Large Language Models (LLMs) have demonstrated proficiency in addressing
tasks that necessitate a combination of task planning and the usage of external
tools that require a blend of task planning and the utilization of external
tools, such as APIs. However, real-world complex systems present three
prevalent challenges concerning task planning and tool usage: (1) The real
system usually has a vast array of APIs, so it is impossible to feed the
descriptions of all APIs to the prompt of LLMs as the token length is limited;
(2) the real system is designed for handling complex tasks, and the base LLMs
can hardly plan a correct sub-task order and API-calling order for such tasks;
(3) Similar semantics and functionalities among APIs in real systems create
challenges for both LLMs and even humans in distinguishing between them. In
response, this paper introduces a comprehensive framework aimed at enhancing
the Task Planning and Tool Usage (TPTU) abilities of LLM-based agents operating
within real-world systems. Our framework comprises three key components
designed to address these challenges: (1) the API Retriever selects the most
pertinent APIs for the user task among the extensive array available; (2) LLM
Finetuner tunes a base LLM so that the finetuned LLM can be more capable for
task planning and API calling; (3) the Demo Selector adaptively retrieves
different demonstrations related to hard-to-distinguish APIs, which is further
used for in-context learning to boost the final performance. We validate our
methods using a real-world commercial system as well as an open-sourced
academic dataset, and the outcomes clearly showcase the efficacy of each
individual component as well as the integrated framework.
A framework is introduced to improve the task planning and tool usage capabilities of LLM-based agents in real-world systems through API retrieval, LLM fine-tuning, and demonstration selection.
Large Language Models (LLMs) API Retriever LLM Finetuner Demo Selector task planning tool usage APIs token length sub-task order API-calling order in-context learning