title: TPTU-v2: Boosting Task Planning and Tool Usage of Large Language
    Model-based Agents in Real-world Systems
summary: Large Language Models (LLMs) have demonstrated proficiency in addressing
  tasks that necessitate a combination of task planning and the usage of external
  tools that require a blend of task planning and the utilization of external
  tools, such as APIs. However, real-world complex systems present three
  prevalent challenges concerning task planning and tool usage: (1) The real
  system usually has a vast array of APIs, so it is impossible to feed the
  descriptions of all APIs to the prompt of LLMs as the token length is limited;
  (2) the real system is designed for handling complex tasks, and the base LLMs
  can hardly plan a correct sub-task order and API-calling order for such tasks;
  (3) Similar semantics and functionalities among APIs in real systems create
  challenges for both LLMs and even humans in distinguishing between them. In
  response, this paper introduces a comprehensive framework aimed at enhancing
  the Task Planning and Tool Usage (TPTU) abilities of LLM-based agents operating
  within real-world systems. Our framework comprises three key components
  designed to address these challenges: (1) the API Retriever selects the most
  pertinent APIs for the user task among the extensive array available; (2) LLM
  Finetuner tunes a base LLM so that the finetuned LLM can be more capable for
  task planning and API calling; (3) the Demo Selector adaptively retrieves
  different demonstrations related to hard-to-distinguish APIs, which is further
  used for in-context learning to boost the final performance. We validate our
  methods using a real-world commercial system as well as an open-sourced
  academic dataset, and the outcomes clearly showcase the efficacy of each
  individual component as well as the integrated framework.
paper_authors._id: 655c2b6971ed242b426b2d4a
paper_authors.name: Yilun Kong
paper_authors._id: 655c2b6971ed242b426b2d4b
paper_authors.name: Jingqing Ruan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T14:08:26.534Z
paper_authors._id: 64872abb3fb9bc8f4994e014
paper_authors.avatarUrl: /avatars/9743b5a2b6f9b9990aa1cb06bd6eb5c6.svg
paper_authors.fullname: Jingqing Ruan
paper_authors.type: user
paper_authors.user: Amanda2023
paper_authors._id: 655c2b6971ed242b426b2d4c
paper_authors.name: Yihong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T14:08:43.117Z
paper_authors._id: 63e09700f0c75dfb876e8570
paper_authors.avatarUrl: /avatars/53c17418163004a6e2730987b1cfc3b8.svg
paper_authors.fullname: Chenyihong
paper_authors.type: user
paper_authors.user: Ffftttlllhappy
paper_authors._id: 655c2b6971ed242b426b2d4d
paper_authors.name: Bin Zhang
paper_authors._id: 655c2b6971ed242b426b2d4e
paper_authors.name: Tianpeng Bao
paper_authors._id: 655c2b6971ed242b426b2d4f
paper_authors.name: Shiwei Shi
paper_authors._id: 655c2b6971ed242b426b2d50
paper_authors.name: Guoqing Du
paper_authors._id: 655c2b6971ed242b426b2d51
paper_authors.name: Xiaoru Hu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-12-16T07:43:34.434Z
paper_authors._id: 670f7471fcbfebab8561d343
paper_authors.avatarUrl: /avatars/79ea08c9d4c359afdf935da107e0b64f.svg
paper_authors.fullname: XR
paper_authors.type: user
paper_authors.user: HU0016
paper_authors._id: 655c2b6971ed242b426b2d52
paper_authors.name: Hangyu Mao
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-11-22T02:13:27.732Z
paper_authors._id: 655d62c5ea26da0aca92746d
paper_authors.avatarUrl: /avatars/74144ab0d0c5c23769cfc2f33ad87a7a.svg
paper_authors.fullname: Hangyu Mao
paper_authors.type: user
paper_authors.user: HangyuMao
paper_authors._id: 655c2b6971ed242b426b2d53
paper_authors.name: Ziyue Li
paper_authors._id: 655c2b6971ed242b426b2d54
paper_authors.name: Xingyu Zeng
paper_authors._id: 655c2b6971ed242b426b2d55
paper_authors.name: Rui Zhao
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-11-21T04:00:42.007Z
paper_authors._id: 6405d7cca577649430bcceb9
paper_authors.avatarUrl: /avatars/befc09d3bc82685c8e205dcadfec13c0.svg
paper_authors.fullname: Rui Zhao
paper_authors.type: user
paper_authors.user: Robert0812
paper_title: TPTU-v2: Boosting Task Planning and Tool Usage of Large Language
    Model-based Agents in Real-world Systems
paper_summary: Large Language Models (LLMs) have demonstrated proficiency in addressing
  tasks that necessitate a combination of task planning and the usage of external
  tools that require a blend of task planning and the utilization of external
  tools, such as APIs. However, real-world complex systems present three
  prevalent challenges concerning task planning and tool usage: (1) The real
  system usually has a vast array of APIs, so it is impossible to feed the
  descriptions of all APIs to the prompt of LLMs as the token length is limited;
  (2) the real system is designed for handling complex tasks, and the base LLMs
  can hardly plan a correct sub-task order and API-calling order for such tasks;
  (3) Similar semantics and functionalities among APIs in real systems create
  challenges for both LLMs and even humans in distinguishing between them. In
  response, this paper introduces a comprehensive framework aimed at enhancing
  the Task Planning and Tool Usage (TPTU) abilities of LLM-based agents operating
  within real-world systems. Our framework comprises three key components
  designed to address these challenges: (1) the API Retriever selects the most
  pertinent APIs for the user task among the extensive array available; (2) LLM
  Finetuner tunes a base LLM so that the finetuned LLM can be more capable for
  task planning and API calling; (3) the Demo Selector adaptively retrieves
  different demonstrations related to hard-to-distinguish APIs, which is further
  used for in-context learning to boost the final performance. We validate our
  methods using a real-world commercial system as well as an open-sourced
  academic dataset, and the outcomes clearly showcase the efficacy of each
  individual component as well as the integrated framework.
paper_ai_keywords: API Retriever LLM Finetuner Demo Selector in-context learning task planning API calling base LLMs