one of the most challenging problem in open-source LLMs. Wei et al. [67] enhances the reasoning
ability of LLMs by augmenting the output with a sequence of intermediate steps toward the answer.
A few methods [18, 66, 74] are proposed to improve the quality of reasoning paths. For example,
Complexity-based CoT [18] selects examples with more steps as in-context demonstrations and shows
that prompting with more reasoning steps leads to better performance. Self-Consistency [66] samples
multiple reasoning paths and selects the final answer by majority voting. Another category of work is
finetuning-based methods, which finetunes open-source models (e.g., LLaMA) with the knowledge
from some advanced closed-source LLMs [46, 48]. Magister et al. [40] investigates the transfer of
reasoning capabilities via knowledge distillation. Yuan et al. [69] proposes to apply rejection sampling
finetuning (RFT) to improve mathematical reasoning performance. WizardMath [38] proposes a
METAMATH