title: Efficient Tool Use with Chain-of-Abstraction Reasoning
summary: To achieve faithful reasoning that aligns with human expectations, large
  language models (LLMs) need to ground their reasoning to real-world knowledge
  (e.g., web facts, math and physical rules). Tools help LLMs access this
  external knowledge, but there remains challenges for fine-tuning LLM agents
  (e.g., Toolformer) to invoke tools in multi-step reasoning problems, where
  inter-connected tool calls require holistic and efficient tool usage planning.
    In this work, we propose a new method for LLMs to better leverage tools in
  multi-step reasoning. Our method, Chain-of-Abstraction (CoA), trains LLMs to
  first decode reasoning chains with abstract placeholders, and then call domain
  tools to reify each reasoning chain by filling in specific knowledge. This
  planning with abstract chains enables LLMs to learn more general reasoning
  strategies, which are robust to shifts of domain knowledge (e.g., math results)
  relevant to different reasoning questions. It also allows LLMs to perform
  decoding and calling of external tools in parallel, which avoids the inference
  delay caused by waiting for tool responses. In mathematical reasoning and Wiki
  QA domains, we show that our method consistently outperforms previous
  chain-of-thought and tool-augmented baselines on both in-distribution and
  out-of-distribution test sets, with an average ~6% absolute QA accuracy
  improvement. LLM agents trained with our method also show more efficient tool
  use, with inference speed being on average ~1.4x faster than baseline
  tool-augmented LLMs.
paper_authors._id: 65bb0b7d00e143d672a7809a
paper_authors.name: Silin Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:23:35.393Z
paper_authors._id: 64b72786f53ae848e71af455
paper_authors.avatarUrl: /avatars/b69326b9e5b3c0ef4f67010f9aa100f2.svg
paper_authors.fullname: Silin Gao
paper_authors.type: user
paper_authors.user: Silin159
paper_authors._id: 65bb0b7d00e143d672a7809b
paper_authors.name: Jane Dwivedi-Yu
paper_authors._id: 65bb0b7d00e143d672a7809c
paper_authors.name: Ping Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:24:02.909Z
paper_authors._id: 646c004e31968a60a022590e
paper_authors.avatarUrl: /avatars/349e2ed4feef1be050fd9903dccec1e1.svg
paper_authors.fullname: Ping Yu
paper_authors.type: user
paper_authors.user: pingyu
paper_authors._id: 65bb0b7d00e143d672a7809d
paper_authors.name: Xiaoqing Ellen Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:24:40.357Z
paper_authors._id: 6345ab8ffe134dfd7a0d1059
paper_authors.avatarUrl: /avatars/159a364ac4f8a2ed001346d4cbb7f2e3.svg
paper_authors.fullname: Ellen Tan
paper_authors.type: user
paper_authors.user: ellenxtan
paper_authors._id: 65bb0b7d00e143d672a7809e
paper_authors.name: Ramakanth Pasunuru
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:24:49.442Z
paper_authors._id: 63b4be2334df1f9a90a1983c
paper_authors.avatarUrl: /avatars/9cdfa3492c87aabdacab684b80628f14.svg
paper_authors.fullname: Ramakanth Pasunuru
paper_authors.type: user
paper_authors.user: rpasunuru
paper_authors._id: 65bb0b7d00e143d672a7809f
paper_authors.name: Olga Golovneva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:24:56.136Z
paper_authors._id: 6318b1e06e6bd039440871d6
paper_authors.avatarUrl: /avatars/2d9f96eb4092a9d1734dcc09303b259a.svg
paper_authors.fullname: Olga Golovneva
paper_authors.type: user
paper_authors.user: Golovneva
paper_authors._id: 65bb0b7d00e143d672a780a0
paper_authors.name: Koustuv Sinha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:25:02.849Z
paper_authors._id: 622b93067b9143726fbedc37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1647023505150-622b93067b9143726fbedc37.jpeg
paper_authors.fullname: Koustuv Sinha
paper_authors.type: user
paper_authors.user: koustuvs
paper_authors._id: 65bb0b7d00e143d672a780a1
paper_authors.name: Asli Celikyilmaz
paper_authors._id: 65bb0b7d00e143d672a780a2
paper_authors.name: Antoine Bosselut
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:26:57.536Z
paper_authors._id: 654fb1c67490049d621516fb
paper_authors.avatarUrl: /avatars/0f88177da236b1bce4a4a58fd0246884.svg
paper_authors.fullname: Antoine Bosselut
paper_authors.type: user
paper_authors.user: atcbosselut
paper_authors._id: 65bb0b7d00e143d672a780a3
paper_authors.name: Tianlu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-01T11:27:04.110Z
paper_authors._id: 63f6a0e6b4c9a104f4b95d86
paper_authors.avatarUrl: /avatars/660c3e44bea976f3e0560bf95466b03c.svg
paper_authors.fullname: Tianlu Wang
paper_authors.type: user
paper_authors.user: Tianlu
paper_title: Efficient Tool Use with Chain-of-Abstraction Reasoning
paper_summary: To achieve faithful reasoning that aligns with human expectations, large
  language models (LLMs) need to ground their reasoning to real-world knowledge
  (e.g., web facts, math and physical rules). Tools help LLMs access this
  external knowledge, but there remains challenges for fine-tuning LLM agents
  (e.g., Toolformer) to invoke tools in multi-step reasoning problems, where
  inter-connected tool calls require holistic and efficient tool usage planning.
    In this work, we propose a new method for LLMs to better leverage tools in
  multi-step reasoning. Our method, Chain-of-Abstraction (CoA), trains LLMs to
  first decode reasoning chains with abstract placeholders, and then call domain
  tools to reify each reasoning chain by filling in specific knowledge. This
  planning with abstract chains enables LLMs to learn more general reasoning
  strategies, which are robust to shifts of domain knowledge (e.g., math results)
  relevant to different reasoning questions. It also allows LLMs to perform
  decoding and calling of external tools in parallel, which avoids the inference
  delay caused by waiting for tool responses. In mathematical reasoning and Wiki
  QA domains, we show that our method consistently outperforms previous
  chain-of-thought and tool-augmented baselines on both in-distribution and
  out-of-distribution test sets, with an average ~6% absolute QA accuracy
  improvement. LLM agents trained with our method also show more efficient tool
  use, with inference speed being on average ~1.4x faster than baseline
  tool-augmented LLMs.
paper_ai_keywords: large language models (LLMs) real-world knowledge web facts math and physical rules tools fine-tuning LLM agents vectorformer Chain-of-Abstraction (CoA) reasoning chains abstract placeholders domain tools holistic and efficient tool usage planning chain-of-thought tool-augmented baselines mathematical reasoning Wiki QA domains decoding and calling of external tools inference speed