title: The ART of LLM Refinement: Ask, Refine, and Trust
summary: In recent years, Large Language Models (LLMs) have demonstrated remarkable
  generative abilities, but can they judge the quality of their own generations?
  A popular concept, referred to as self-refinement, postulates that LLMs can
  detect and correct the errors in their generations when asked to do so.
  However, recent empirical evidence points in the opposite direction, suggesting
  that LLMs often struggle to accurately identify errors when reasoning is
  involved. To address this, we propose a reasoning with refinement objective
  called ART: Ask, Refine, and Trust, which asks necessary questions to decide
  when an LLM should refine its output, and either affirm or withhold trust in
  its refinement by ranking the refinement and the initial prediction. On two
  multistep reasoning tasks of mathematical word problems (GSM8K) and question
  answering (StrategyQA), ART achieves a performance gain of +5 points over
  self-refinement baselines, while using a much smaller model as the decision
  maker. We also demonstrate the benefit of using smaller models to make
  refinement decisions as a cost-effective alternative to fine-tuning a larger
  model.
paper_authors._id: 655439a835a2204c34ec0f29
paper_authors.name: Kumar Shridhar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-15T10:17:40.430Z
paper_authors._id: 64f3bdddb05bf790b63b6116
paper_authors.avatarUrl: /avatars/c181133a9d8a6a75d706c34d921784c1.svg
paper_authors.fullname: Kumar Shridhar
paper_authors.type: user
paper_authors.user: shirdhar743-meta
paper_authors._id: 655439a835a2204c34ec0f2a
paper_authors.name: Koustuv Sinha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-15T10:17:47.858Z
paper_authors._id: 622b93067b9143726fbedc37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1647023505150-622b93067b9143726fbedc37.jpeg
paper_authors.fullname: Koustuv Sinha
paper_authors.type: user
paper_authors.user: koustuvs
paper_authors._id: 655439a835a2204c34ec0f2b
paper_authors.name: Andrew Cohen
paper_authors._id: 655439a835a2204c34ec0f2c
paper_authors.name: Tianlu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-15T10:18:14.136Z
paper_authors._id: 63f6a0e6b4c9a104f4b95d86
paper_authors.avatarUrl: /avatars/660c3e44bea976f3e0560bf95466b03c.svg
paper_authors.fullname: Tianlu Wang
paper_authors.type: user
paper_authors.user: Tianlu
paper_authors._id: 655439a835a2204c34ec0f2d
paper_authors.name: Ping Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-15T10:18:28.515Z
paper_authors._id: 646c004e31968a60a022590e
paper_authors.avatarUrl: /avatars/349e2ed4feef1be050fd9903dccec1e1.svg
paper_authors.fullname: Ping Yu
paper_authors.type: user
paper_authors.user: pingyu
paper_authors._id: 655439a835a2204c34ec0f2e
paper_authors.name: Ram Pasunuru
paper_authors._id: 655439a835a2204c34ec0f2f
paper_authors.name: Mrinmaya Sachan
paper_authors._id: 655439a835a2204c34ec0f30
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-15T10:18:49.386Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 655439a835a2204c34ec0f31
paper_authors.name: Asli Celikyilmaz
paper_title: The ART of LLM Refinement: Ask, Refine, and Trust
paper_summary: In recent years, Large Language Models (LLMs) have demonstrated remarkable
  generative abilities, but can they judge the quality of their own generations?
  A popular concept, referred to as self-refinement, postulates that LLMs can
  detect and correct the errors in their generations when asked to do so.
  However, recent empirical evidence points in the opposite direction, suggesting
  that LLMs often struggle to accurately identify errors when reasoning is
  involved. To address this, we propose a reasoning with refinement objective
  called ART: Ask, Refine, and Trust, which asks necessary questions to decide
  when an LLM should refine its output, and either affirm or withhold trust in
  its refinement by ranking the refinement and the initial prediction. On two
  multistep reasoning tasks of mathematical word problems (GSM8K) and question
  answering (StrategyQA), ART achieves a performance gain of +5 points over
  self-refinement baselines, while using a much smaller model as the decision
  maker. We also demonstrate the benefit of using smaller models to make
  refinement decisions as a cost-effective alternative to fine-tuning a larger
  model.
paper_ai_summary: The ART method improves the refinement of Large Language Model (LLM) outputs on multistep reasoning tasks by using smaller models to decide when and how to refine the initial predictions.
paper_ai_keywords: Large Language Models self-refinement reasoning with refinement ART mathematical word problems question answering GSM8K StrategyQA decision maker smaller models