3

Technical Report

3 METHOD

The overview of our method is illustrated in Figure 1. Given a meta-question (a sample in the original
mathematical training set), we can generate a series of variants. Specifically, we perform three types
of question bootstrapping. Combined with answer augmentation, we present MetaMathQA, a diverse
and high-quality mathematical dataset based on GSM8K and MATH. We then present MetaMath, a
family of LLMs finetuned on MetaMathQA focusing on elementary mathematical problem-solving.

3.1 ANSWER AUGMENTATION (ANSAUG)

Generating more reasoning paths is a simple but effective way to augment the training set. For a
question qi, we use few-shot chain-of-thought prompting with temperature sampling to generate
KAnsAug more reasoning paths {(r(j)
i ) : j = 1, . . . , KAnsAug}: the question is appended to a few
, a(j)
in-context reasoning examples, then fed to the LLM for generating its reasoning path r(j)
and answer
a(j)
i
METAMATH