title: AgentTuning: Enabling Generalized Agent Abilities for LLMs
summary: Open large language models (LLMs) with great performance in various tasks
  have significantly advanced the development of LLMs. However, they are far
  inferior to commercial models such as ChatGPT and GPT-4 when acting as agents
  to tackle complex tasks in the real world. These agent tasks employ LLMs as the
  central controller responsible for planning, memorization, and tool
  utilization, necessitating both fine-grained prompting methods and robust LLMs
  to achieve satisfactory performance. Though many prompting methods have been
  proposed to complete particular agent tasks, there is lack of research focusing
  on improving the agent capabilities of LLMs themselves without compromising
  their general abilities. In this work, we present AgentTuning, a simple and
  general method to enhance the agent abilities of LLMs while maintaining their
  general LLM capabilities. We construct AgentInstruct, a lightweight
  instruction-tuning dataset containing high-quality interaction trajectories. We
  employ a hybrid instruction-tuning strategy by combining AgentInstruct with
  open-source instructions from general domains. AgentTuning is used to
  instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show
  that AgentTuning enables LLMs' agent capabilities without compromising general
  abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent
  tasks, demonstrating generalized agent capabilities. We open source the
  AgentInstruct and AgentLM-7B, 13B, and 70B models at
  https://github.com/THUDM/AgentTuning , serving open and powerful alternatives
  to commercial LLMs for agent tasks.
paper_authors._id: 6531dc81e7897259333f12d1
paper_authors.name: Aohan Zeng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-20T08:05:59.721Z
paper_authors._id: 62dc173789b4cf157d36ebee
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1659772051636-62dc173789b4cf157d36ebee.jpeg
paper_authors.fullname: Zeng Aohan
paper_authors.type: user
paper_authors.user: Sengxian
paper_authors._id: 6531dc81e7897259333f12d2
paper_authors.name: Mingdao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:24:04.545Z
paper_authors._id: 64f1abd2b12bdcef55e1c078
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f1abd2b12bdcef55e1c078/yG9HR_imUWqESN-JNcjf1.jpeg
paper_authors.fullname: Mingdao Liu
paper_authors.type: user
paper_authors.user: lambdax
paper_authors._id: 6531dc81e7897259333f12d3
paper_authors.name: Rui Lu
paper_authors._id: 6531dc81e7897259333f12d4
paper_authors.name: Bowen Wang
paper_authors._id: 6531dc81e7897259333f12d5
paper_authors.name: Xiao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:28:18.151Z
paper_authors._id: 62f098602ca4d32a7cd87aba
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1659934844914-62f098602ca4d32a7cd87aba.jpeg
paper_authors.fullname: Xiao Liu
paper_authors.type: user
paper_authors.user: ShawLiu
paper_authors._id: 6531dc81e7897259333f12d6
paper_authors.name: Yuxiao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:25:37.715Z
paper_authors._id: 640e73bdfdeaae1390857b62
paper_authors.avatarUrl: /avatars/cd6779e30f716002a7838ed93d5c0754.svg
paper_authors.fullname: Yuxiao Dong
paper_authors.type: user
paper_authors.user: yuxiaod
paper_authors._id: 6531dc81e7897259333f12d7
paper_authors.name: Jie Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:25:53.463Z
paper_authors._id: 640dff05474aa6f89556677e
paper_authors.avatarUrl: /avatars/1b4591c7322d649c797b3125148f1915.svg
paper_authors.fullname: Jie Tang
paper_authors.type: user
paper_authors.user: jerytang
paper_title: AgentTuning: Enabling Generalized Agent Abilities for LLMs
paper_summary: Open large language models (LLMs) with great performance in various tasks
  have significantly advanced the development of LLMs. However, they are far
  inferior to commercial models such as ChatGPT and GPT-4 when acting as agents
  to tackle complex tasks in the real world. These agent tasks employ LLMs as the
  central controller responsible for planning, memorization, and tool
  utilization, necessitating both fine-grained prompting methods and robust LLMs
  to achieve satisfactory performance. Though many prompting methods have been
  proposed to complete particular agent tasks, there is lack of research focusing
  on improving the agent capabilities of LLMs themselves without compromising
  their general abilities. In this work, we present AgentTuning, a simple and
  general method to enhance the agent abilities of LLMs while maintaining their
  general LLM capabilities. We construct AgentInstruct, a lightweight
  instruction-tuning dataset containing high-quality interaction trajectories. We
  employ a hybrid instruction-tuning strategy by combining AgentInstruct with
  open-source instructions from general domains. AgentTuning is used to
  instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show
  that AgentTuning enables LLMs' agent capabilities without compromising general
  abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent
  tasks, demonstrating generalized agent capabilities. We open source the
  AgentInstruct and AgentLM-7B, 13B, and 70B models at
  https://github.com/THUDM/AgentTuning , serving open and powerful alternatives
  to commercial LLMs for agent tasks.
paper_ai_summary: AgentTuning enhances the agent capabilities of open LLMs through instruction-tuning without compromising their general abilities, demonstrating performance comparable to commercial models on unseen tasks.
paper_ai_keywords: LLMs AgentTuning AgentInstruct instruction-tuning AgentLM