The ART of LLM Refinement: Ask, Refine, and Trust
In recent years, Large Language Models (LLMs) have demonstrated remarkable
generative abilities, but can they judge the quality of their own generations?
A popular concept, referred to as self-refinement, postulates that LLMs can
detect and correct the errors in their generations when asked to do so.
However, recent empirical evidence points in the opposite direction, suggesting
that LLMs often struggle to accurately identify errors when reasoning is
involved. To address this, we propose a reasoning with refinement objective
called ART: Ask, Refine, and Trust, which asks necessary questions to decide
when an LLM should refine its output, and either affirm or withhold trust in
its refinement by ranking the refinement and the initial prediction. On two
multistep reasoning tasks of mathematical word problems (GSM8K) and question
answering (StrategyQA), ART achieves a performance gain of +5 points over
self-refinement baselines, while using a much smaller model as the decision
maker. We also demonstrate the benefit of using smaller models to make
refinement decisions as a cost-effective alternative to fine-tuning a larger
model.
655439a835a2204c34ec0f29
Kumar Shridhar
admin_assigned
2023-11-15T10:17:40.430Z
64f3bdddb05bf790b63b6116
/avatars/c181133a9d8a6a75d706c34d921784c1.svg
Kumar Shridhar
user
shirdhar743-meta
655439a835a2204c34ec0f2a
Koustuv Sinha
admin_assigned
2023-11-15T10:17:47.858Z
622b93067b9143726fbedc37
https://cdn-avatars.huggingface.co/v1/production/uploads/1647023505150-622b93067b9143726fbedc37.jpeg
Koustuv Sinha
user
koustuvs
655439a835a2204c34ec0f2b
Andrew Cohen
655439a835a2204c34ec0f2c
Tianlu Wang
admin_assigned
2023-11-15T10:18:14.136Z
63f6a0e6b4c9a104f4b95d86
/avatars/660c3e44bea976f3e0560bf95466b03c.svg
Tianlu Wang
user
Tianlu
655439a835a2204c34ec0f2d
Ping Yu
admin_assigned
2023-11-15T10:18:28.515Z
646c004e31968a60a022590e
/avatars/349e2ed4feef1be050fd9903dccec1e1.svg
Ping Yu
user
pingyu
655439a835a2204c34ec0f2e
Ram Pasunuru
655439a835a2204c34ec0f2f
Mrinmaya Sachan
655439a835a2204c34ec0f30
Jason Weston
admin_assigned
2023-11-15T10:18:49.386Z
62f023a36a027498eaa2f9cc
/avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
Jason Weston
user
spermwhale
655439a835a2204c34ec0f31
Asli Celikyilmaz
The ART of LLM Refinement: Ask, Refine, and Trust
In recent years, Large Language Models (LLMs) have demonstrated remarkable
generative abilities, but can they judge the quality of their own generations?
A popular concept, referred to as self-refinement, postulates that LLMs can
detect and correct the errors in their generations when asked to do so.
However, recent empirical evidence points in the opposite direction, suggesting
that LLMs often struggle to accurately identify errors when reasoning is
involved. To address this, we propose a reasoning with refinement objective
called ART: Ask, Refine, and Trust, which asks necessary questions to decide
when an LLM should refine its output, and either affirm or withhold trust in
its refinement by ranking the refinement and the initial prediction. On two
multistep reasoning tasks of mathematical word problems (GSM8K) and question
answering (StrategyQA), ART achieves a performance gain of +5 points over
self-refinement baselines, while using a much smaller model as the decision
maker. We also demonstrate the benefit of using smaller models to make
refinement decisions as a cost-effective alternative to fine-tuning a larger
model.
The ART method improves the refinement of Large Language Model (LLM) outputs on multistep reasoning tasks by using smaller models to decide when and how to refine the initial predictions.
Large Language Models self-refinement reasoning with refinement ART mathematical word problems question answering GSM8K StrategyQA decision maker smaller models