mathematical tasks.
We also conduct an experiment by fine-
tuning LLaMA-2-7B on the MetaMathQA-
MATH data then evaluate the model on
GSM8K and MATH. Table 1 shows the
testing accuracy. Again, MetaMath trained
on AnsAug or rephrasing augmentation
data performs much better than SFT. Fur-
thermore, merging all augmented data to-
gether for fine-tuning is better than merg-
ing AnsAug and rephrasing augmentation
data, demonstrating the effectiveness of SV
and FOBAR augmentation data in improv-
ing mathematical reasoning ability. More-
over, for the unseen GSM8K task, Meta-
Math trained on MetaMathQA-MATH data
is significantly better than SFT (+20%).
METAMATH