title: LiteSearch: Efficacious Tree Search for LLM
summary: Recent research suggests that tree search algorithms (e.g. Monte Carlo Tree
  Search) can dramatically boost LLM performance on complex mathematical
  reasoning tasks. However, they often require more than 10 times the
  computational resources of greedy decoding due to wasteful search strategies,
  making them difficult to be deployed in practical applications. This study
  introduces a novel guided tree search algorithm with dynamic node selection and
  node-level exploration budget (maximum number of children) calculation to
  tackle this issue. By considering the search progress towards the final answer
  (history) and the guidance from a value network (future) trained without any
  step-wise annotations, our algorithm iteratively selects the most promising
  tree node before expanding it within the boundaries of the allocated
  computational budget. Experiments conducted on the GSM8K and TabMWP datasets
  demonstrate that our approach not only offers competitive performance but also
  enjoys significantly lower computational costs compared to baseline methods.
paper_authors._id: 668371213d01a30be915a776
paper_authors.name: Ante Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:37:14.804Z
paper_authors._id: 657814faa87010c9f87cc6d4
paper_authors.avatarUrl: /avatars/764cf156bacd07c8268702e5b92a478f.svg
paper_authors.fullname: Ante Wang
paper_authors.type: user
paper_authors.user: xmu-nlp
paper_authors._id: 668371213d01a30be915a777
paper_authors.name: Linfeng Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T09:19:37.386Z
paper_authors._id: 64c94eddcb2f1bf0e7db5a4d
paper_authors.avatarUrl: /avatars/f7e2532d3c85d5e5b5a02c579ea68c3a.svg
paper_authors.fullname: Linfeng Song
paper_authors.type: user
paper_authors.user: freesunshine0316
paper_authors._id: 668371213d01a30be915a778
paper_authors.name: Ye Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-17T11:29:08.511Z
paper_authors._id: 6414061f996b2e426f23df48
paper_authors.avatarUrl: /avatars/db0af49c5f017f7c455a076b8b2212b6.svg
paper_authors.fullname: Ye Tian
paper_authors.type: user
paper_authors.user: yetian
paper_authors._id: 668371213d01a30be915a779
paper_authors.name: Baolin Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T09:25:52.770Z
paper_authors._id: 61942296d5c2ba6daa290357
paper_authors.avatarUrl: /avatars/594021cc183c4922d48b46f43772a062.svg
paper_authors.fullname: Baolin Peng
paper_authors.type: user
paper_authors.user: Baolin
paper_authors._id: 668371213d01a30be915a77a
paper_authors.name: Dian Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-03T07:41:10.324Z
paper_authors._id: 62d58fd53bf5e059f7cc3245
paper_authors.avatarUrl: /avatars/7a4f3ee4a37245f67efd26749d66a706.svg
paper_authors.fullname: Dian Yu
paper_authors.type: user
paper_authors.user: yudian
paper_authors._id: 668371213d01a30be915a77b
paper_authors.name: Haitao Mi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T09:26:00.192Z
paper_authors._id: 65147a1426fbd558dbd08f1b
paper_authors.avatarUrl: /avatars/86574ee2d5c22e940be1c4e50be88675.svg
paper_authors.fullname: Haitao Mi
paper_authors.type: user
paper_authors.user: haitaominlp
paper_authors._id: 668371213d01a30be915a77c
paper_authors.name: Jinsong Su
paper_authors._id: 668371213d01a30be915a77d
paper_authors.name: Dong Yu
paper_title: LiteSearch: Efficacious Tree Search for LLM
paper_summary: Recent research suggests that tree search algorithms (e.g. Monte Carlo Tree
  Search) can dramatically boost LLM performance on complex mathematical
  reasoning tasks. However, they often require more than 10 times the
  computational resources of greedy decoding due to wasteful search strategies,
  making them difficult to be deployed in practical applications. This study
  introduces a novel guided tree search algorithm with dynamic node selection and
  node-level exploration budget (maximum number of children) calculation to
  tackle this issue. By considering the search progress towards the final answer
  (history) and the guidance from a value network (future) trained without any
  step-wise annotations, our algorithm iteratively selects the most promising
  tree node before expanding it within the boundaries of the allocated
  computational budget. Experiments conducted on the GSM8K and TabMWP datasets
  demonstrate that our approach not only offers competitive performance but also
  enjoys significantly lower computational costs compared to baseline methods.
paper_ai_keywords: Monte Carlo Tree Search tree search algorithms LLM performance complex mathematical reasoning tasks computational resources greedy decoding guided tree search algorithm dynamic node selection node-level exploration budget search strategies value network computational budget GSM8K dataset TabMWP dataset competitive performance lower computational costs