title: Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
summary: Language agents perform complex tasks by using tools to execute each step
  precisely. However, most existing agents are based on proprietary models or
  designed to target specific tasks, such as mathematics or multi-hop question
  answering. We introduce Husky, a holistic, open-source language agent that
  learns to reason over a unified action space to address a diverse set of
  complex tasks involving numerical, tabular, and knowledge-based reasoning.
  Husky iterates between two stages: 1) generating the next action to take
  towards solving a given task and 2) executing the action using expert models
  and updating the current solution state. We identify a thorough ontology of
  actions for addressing complex tasks and curate high-quality data to train
  expert models for executing these actions. Our experiments show that Husky
  outperforms prior language agents across 14 evaluation datasets. Moreover, we
  introduce HuskyQA, a new evaluation set which stress tests language agents for
  mixed-tool reasoning, with a focus on retrieving missing knowledge and
  performing numerical reasoning. Despite using 7B models, Husky matches or even
  exceeds frontier LMs such as GPT-4 on these tasks, showcasing the efficacy of
  our holistic approach in addressing complex reasoning problems. Our code and
  models are available at https://github.com/agent-husky/Husky-v1.
paper_authors._id: 6667cfedcdac3c8f11acf0a0
paper_authors.name: Joongwon Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:53:14.744Z
paper_authors._id: 64b81bc20d8834cb3225bbc0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b81bc20d8834cb3225bbc0/F2ciNosMyIlC2ZLlVTpw2.jpeg
paper_authors.fullname: Joongwon Kim
paper_authors.type: user
paper_authors.user: jwonkim
paper_authors._id: 6667cfedcdac3c8f11acf0a1
paper_authors.name: Bhargavi Paranjape
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:55:49.678Z
paper_authors._id: 63efd0944a788ed1dd84fd57
paper_authors.avatarUrl: /avatars/db1e0fee54151fcf795b4826d625a332.svg
paper_authors.fullname: Bhargavi Paranjape
paper_authors.type: user
paper_authors.user: bhargaviparanjape
paper_authors._id: 6667cfedcdac3c8f11acf0a2
paper_authors.name: Tushar Khot
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:55:55.579Z
paper_authors._id: 62cdf2dbaac2c91c95581830
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62cdf2dbaac2c91c95581830/N3NuO7j53qkauJOf3kkRS.jpeg
paper_authors.fullname: Tushar Khot
paper_authors.type: user
paper_authors.user: tusharkhot
paper_authors._id: 6667cfedcdac3c8f11acf0a3
paper_authors.name: Hannaneh Hajishirzi
paper_title: Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
paper_summary: Language agents perform complex tasks by using tools to execute each step
  precisely. However, most existing agents are based on proprietary models or
  designed to target specific tasks, such as mathematics or multi-hop question
  answering. We introduce Husky, a holistic, open-source language agent that
  learns to reason over a unified action space to address a diverse set of
  complex tasks involving numerical, tabular, and knowledge-based reasoning.
  Husky iterates between two stages: 1) generating the next action to take
  towards solving a given task and 2) executing the action using expert models
  and updating the current solution state. We identify a thorough ontology of
  actions for addressing complex tasks and curate high-quality data to train
  expert models for executing these actions. Our experiments show that Husky
  outperforms prior language agents across 14 evaluation datasets. Moreover, we
  introduce HuskyQA, a new evaluation set which stress tests language agents for
  mixed-tool reasoning, with a focus on retrieving missing knowledge and
  performing numerical reasoning. Despite using 7B models, Husky matches or even
  exceeds frontier LMs such as GPT-4 on these tasks, showcasing the efficacy of
  our holistic approach in addressing complex reasoning problems. Our code and
  models are available at https://github.com/agent-husky/Husky-v1.
paper_ai_keywords: holistic, open-source language agent unified action space numerical reasoning tabular reasoning knowledge-based reasoning generating the next action executing the action expert models ontology of actions high-quality data control experiments evaluation datasets mixed-tool reasoning HuskyQA state-of-the-art models GPT-4 VSM model