title: Octo-planner: On-device Language Model for Planner-Action Agents
summary: AI agents have become increasingly significant in various domains, enabling
  autonomous decision-making and problem-solving. To function effectively, these
  agents require a planning process that determines the best course of action and
  then executes the planned actions. In this paper, we present an efficient
  on-device Planner-Action framework that separates planning and action execution
  into two distinct components: a planner agent based on Phi-3 Mini, a 3.8
  billion parameter LLM optimized for edge devices, and an action agent using the
  Octopus model for function execution. The planner agent first responds to user
  queries by decomposing tasks into a sequence of sub-steps, which are then
  executed by the action agent. To optimize performance on resource-constrained
  devices, we employ model fine-tuning instead of in-context learning, reducing
  computational costs and energy consumption while improving response times. Our
  approach involves using GPT-4 to generate diverse planning queries and
  responses based on available functions, with subsequent validations to ensure
  data quality. We fine-tune the Phi-3 Mini model on this curated dataset,
  achieving a 97\% success rate in our in-domain test environment. To address
  multi-domain planning challenges, we developed a multi-LoRA training method
  that merges weights from LoRAs trained on distinct function subsets. This
  approach enables flexible handling of complex, multi-domain queries while
  maintaining computational efficiency on resource-constrained devices. To
  support further research, we have open-sourced our model weights at
  https://huggingface.co/NexaAIDev/octopus-planning. For the demo, please
  refer to https://www.nexa4ai.com/octo-planner.
paper_authors._id: 667cbb5102ffd4993e2611f0
paper_authors.name: Wei Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-28T08:03:10.308Z
paper_authors._id: 65c082c626c639b0009a3c82
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65c082c626c639b0009a3c82/w_Q_5iLOM-nh_drd9Tu4X.jpeg
paper_authors.fullname: Alex Chen
paper_authors.type: user
paper_authors.user: alexchen4ai
paper_authors._id: 667cbb5102ffd4993e2611f1
paper_authors.name: Zhiyuan Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T07:31:33.153Z
paper_authors._id: 644de55076c0ab1880b9cb4d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644de55076c0ab1880b9cb4d/zKm7baOIzt_a4py8mYkck.jpeg
paper_authors.fullname: Zack Li
paper_authors.type: user
paper_authors.user: zackli4ai
paper_authors._id: 667cbb5102ffd4993e2611f2
paper_authors.name: Zhen Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T07:31:30.780Z
paper_authors._id: 638e4e66629b4d0a62ce1bf3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/638e4e66629b4d0a62ce1bf3/s7uQ2qmee2CaXfjZDOovZ.jpeg
paper_authors.fullname: Zhen Guo
paper_authors.type: user
paper_authors.user: zguo0525
paper_authors._id: 667cbb5102ffd4993e2611f3
paper_authors.name: Yikang Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-27T19:08:17.201Z
paper_authors._id: 6463995939359568c63d740c
paper_authors.avatarUrl: /avatars/11caacefc68c1259dbc8764cc4340481.svg
paper_authors.fullname: Yikang Shen
paper_authors.type: user
paper_authors.user: YikangS
paper_title: Octo-planner: On-device Language Model for Planner-Action Agents
paper_summary: AI agents have become increasingly significant in various domains, enabling
  autonomous decision-making and problem-solving. To function effectively, these
  agents require a planning process that determines the best course of action and
  then executes the planned actions. In this paper, we present an efficient
  on-device Planner-Action framework that separates planning and action execution
  into two distinct components: a planner agent based on Phi-3 Mini, a 3.8
  billion parameter LLM optimized for edge devices, and an action agent using the
  Octopus model for function execution. The planner agent first responds to user
  queries by decomposing tasks into a sequence of sub-steps, which are then
  executed by the action agent. To optimize performance on resource-constrained
  devices, we employ model fine-tuning instead of in-context learning, reducing
  computational costs and energy consumption while improving response times. Our
  approach involves using GPT-4 to generate diverse planning queries and
  responses based on available functions, with subsequent validations to ensure
  data quality. We fine-tune the Phi-3 Mini model on this curated dataset,
  achieving a 97\% success rate in our in-domain test environment. To address
  multi-domain planning challenges, we developed a multi-LoRA training method
  that merges weights from LoRAs trained on distinct function subsets. This
  approach enables flexible handling of complex, multi-domain queries while
  maintaining computational efficiency on resource-constrained devices. To
  support further research, we have open-sourced our model weights at
  https://huggingface.co/NexaAIDev/octopus-planning. For the demo, please
  refer to https://www.nexa4ai.com/octo-planner.
paper_ai_summary: An on-device Planner-Action framework, utilizing Phi-3 Mini for planning and Octopus for execution, employs model fine-tuning and multi-LoRA training to enhance efficiency and multi-domain handling on resource-constrained devices.
paper_ai_keywords: LLM Phi-3 Mini Octopus model model fine-tuning GPT-4 in-context learning LoRA multi-LoRA training on-device framework computational efficiency