title: DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning
    Trajectories Search
summary: Enhancing the capability of large language models (LLMs) in reasoning has
  gained significant attention in recent years. Previous studies have
  demonstrated the effectiveness of various prompting strategies in aiding LLMs
  in reasoning (called "reasoning actions"), such as step-by-step thinking,
  reflecting before answering, solving with programs, and their combinations.
  However, these approaches often applied static, predefined reasoning actions
  uniformly to all questions, without considering the specific characteristics of
  each question or the capability of the task-solving LLM. In this paper, we
  propose DOTS, an approach enabling LLMs to reason dynamically via optimal
  reasoning trajectory search, tailored to the specific characteristics of each
  question and the inherent capability of the task-solving LLM. Our approach
  involves three key steps: i) defining atomic reasoning action modules that can
  be composed into various reasoning action trajectories; ii) searching for the
  optimal action trajectory for each training question through iterative
  exploration and evaluation for the specific task-solving LLM; and iii) using
  the collected optimal trajectories to train an LLM to plan for the reasoning
  trajectories of unseen questions. In particular, we propose two learning
  paradigms, i.e., fine-tuning an external LLM as a planner to guide the
  task-solving LLM, or directly fine-tuning the task-solving LLM with an
  internalized capability for reasoning actions planning. Our experiments across
  eight reasoning tasks show that our method consistently outperforms static
  reasoning techniques and the vanilla instruction tuning approach. Further
  analysis reveals that our method enables LLMs to adjust their computation based
  on problem complexity, allocating deeper thinking and reasoning to harder
  problems.
paper_authors._id: 6706c8cc10d2faa686333496
paper_authors.name: Murong Yue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:25:50.840Z
paper_authors._id: 63e1ac9f7fbb6ae4d4f312a3
paper_authors.avatarUrl: /avatars/7d999543d3ca3df3fc8bb96cd3a3f8cb.svg
paper_authors.fullname: Murong Yue
paper_authors.type: user
paper_authors.user: MurongYue
paper_authors._id: 6706c8cc10d2faa686333497
paper_authors.name: Wenlin Yao
paper_authors._id: 6706c8cc10d2faa686333498
paper_authors.name: Haitao Mi
paper_authors._id: 6706c8cc10d2faa686333499
paper_authors.name: Dian Yu
paper_authors._id: 6706c8cc10d2faa68633349a
paper_authors.name: Ziyu Yao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T15:50:35.621Z
paper_authors._id: 62d87e143cd901df9446a37a
paper_authors.avatarUrl: /avatars/b9a7d24bbe6e49ac90902fafee3a74e7.svg
paper_authors.fullname: Ziyu Yao
paper_authors.type: user
paper_authors.user: ziyuyao
paper_authors._id: 6706c8cc10d2faa68633349b
paper_authors.name: Dong Yu
paper_title: DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning
    Trajectories Search
paper_summary: Enhancing the capability of large language models (LLMs) in reasoning has
  gained significant attention in recent years. Previous studies have
  demonstrated the effectiveness of various prompting strategies in aiding LLMs
  in reasoning (called "reasoning actions"), such as step-by-step thinking,
  reflecting before answering, solving with programs, and their combinations.
  However, these approaches often applied static, predefined reasoning actions
  uniformly to all questions, without considering the specific characteristics of
  each question or the capability of the task-solving LLM. In this paper, we
  propose DOTS, an approach enabling LLMs to reason dynamically via optimal
  reasoning trajectory search, tailored to the specific characteristics of each
  question and the inherent capability of the task-solving LLM. Our approach
  involves three key steps: i) defining atomic reasoning action modules that can
  be composed into various reasoning action trajectories; ii) searching for the
  optimal action trajectory for each training question through iterative
  exploration and evaluation for the specific task-solving LLM; and iii) using
  the collected optimal trajectories to train an LLM to plan for the reasoning
  trajectories of unseen questions. In particular, we propose two learning
  paradigms, i.e., fine-tuning an external LLM as a planner to guide the
  task-solving LLM, or directly fine-tuning the task-solving LLM with an
  internalized capability for reasoning actions planning. Our experiments across
  eight reasoning tasks show that our method consistently outperforms static
  reasoning techniques and the vanilla instruction tuning approach. Further
  analysis reveals that our method enables LLMs to adjust their computation based
  on problem complexity, allocating deeper thinking and reasoning to harder
  problems.
paper_ai_summary: An approach called DOTS enables large language models to dynamically reason by selecting optimal reasoning trajectories tailored to each question, improving performance across various reasoning tasks.
paper_ai_keywords: large language models LLMs reasoning actions step-by-step thinking reflecting solving with programs dynamic reasoning optimal reasoning trajectory search atomic reasoning action modules training question reasoning action trajectories iterative exploration evaluation fine-tuning external LLM internalized capability reasoning actions planning problem complexity deeper thinking