Large language models (LLMs) have accomplished significant breakthroughs in a wide variety
of domains, including mathematical reasoning (Cobbe et al., 2021; Wei et al., 2022; Lewkowycz
et al., 2022), code generation (Chen et al., 2021; Austin et al., 2021; Li et al., 2022), and other text
generation tasks (Bubeck et al., 2023; Anil et al., 2023; Touvron et al., 2023). Despite the rapid
progress, the LLM-generated responses are still prone to errors when they get long. A long line of
efforts have been devoted to improve the output quality by sampling multiple model responses and
then selecting the final output based on certain criteria. For example, prior works have trained neural
networks to rerank model outputs (Cobbe et al., 2021; Li et al., 2023b; Ni et al., 2023; Yin & Neubig,
2019; Zeng et al., 2022), and more recent works investigate using LLMs to score the responses (Fu
et al., 2023; Liu et al., 2023; Wang et al., 2023a).
UNIVERSALSELF-CONSISTENCYFORLARGELANGUAGEMODELGENERATION