Orca-Math: Unlocking the potential of SLMs in Grade School Math
Mathematical word problem-solving has long been recognized as a complex task
for small language models (SLMs). A recent study hypothesized that the smallest
model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34
billion parameters. To reach this level of performance with smaller models,
researcher often train SLMs to generate Python code or use tools to help avoid
calculation errors. Additionally, they employ ensembling, where outputs of up
to 100 model runs are combined to arrive at a more accurate result. Result
selection is done using consensus, majority vote or a separate a verifier model
used in conjunction with the SLM. Ensembling provides a substantial boost in
accuracy but at a significant cost increase with multiple calls to the model
(e.g., Phi-GSM uses top-48 to boost the performance from 68.2 to 81.5).
  In this work, we present Orca-Math, a 7-billion-parameter SLM based on the
Mistral-7B, which achieves 86.81% on GSM8k without the need for multiple model
calls or the use of verifiers, code execution or any other external tools. Our
approach has the following key elements: (1) A high quality synthetic dataset
of 200K math problems created using a multi-agent setup where agents
collaborate to create the data, (2) An iterative learning techniques that
enables the SLM to practice solving problems, receive feedback on its solutions
and learn from preference pairs incorporating the SLM solutions and the
feedback. When trained with Supervised Fine-Tuning alone, Orca-Math achieves
81.50% on GSM8k pass@1 metric. With iterative preference learning, Orca-Math
achieves 86.81% pass@1. Orca-Math surpasses the performance of significantly
larger models such as LLAMA-2-70B, WizardMath-70B, Gemini-Pro, ChatGPT-3.5. It
also significantly outperforms other smaller models while using much smaller
data (hundreds of thousands vs. millions of problems).
65dbf639139bc4eee34a10ff
Arindam Mitra
admin_assigned
2024-02-26T10:19:55.384Z
609ee7bfa4d9efdf9ce888aa
/avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
Arindam Mitra
user
ari9dam
65dbf639139bc4eee34a1100
Hamed Khanpour
admin_assigned
2024-02-26T10:20:01.313Z
6553d3d55ba75868f6651023
/avatars/c052a841e13fb7978905f5c092fbc685.svg
Hamed Khanpour
user
hamedkhanpour
65dbf639139bc4eee34a1101
Corby Rosset
admin_assigned
2024-02-26T10:20:07.586Z
64ec43147e2ec711a761c594
/avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
corby
user
corbyrosset
65dbf639139bc4eee34a1102
Ahmed Awadallah
admin_assigned
2024-02-26T10:20:19.898Z
6557a41ee74d1065f7d05c51
/avatars/b29e57fc61d1d3012637f84b82c14818.svg
Ahmed Awadallah
user
AhmedAwadallah
Orca-Math: Unlocking the potential of SLMs in Grade School Math
Mathematical word problem-solving has long been recognized as a complex task
for small language models (SLMs). A recent study hypothesized that the smallest
model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34
billion parameters. To reach this level of performance with smaller models,
researcher often train SLMs to generate Python code or use tools to help avoid
calculation errors. Additionally, they employ ensembling, where outputs of up
to 100 model runs are combined to arrive at a more accurate result. Result
selection is done using consensus, majority vote or a separate a verifier model
used in conjunction with the SLM. Ensembling provides a substantial boost in
accuracy but at a significant cost increase with multiple calls to the model
(e.g., Phi-GSM uses top-48 to boost the performance from 68.2 to 81.5).
  In this work, we present Orca-Math, a 7-billion-parameter SLM based on the
Mistral-7B, which achieves 86.81% on GSM8k without the need for multiple model
calls or the use of verifiers, code execution or any other external tools. Our
approach has the following key elements: (1) A high quality synthetic dataset
of 200K math problems created using a multi-agent setup where agents
collaborate to create the data, (2) An iterative learning techniques that
enables the SLM to practice solving problems, receive feedback on its solutions
and learn from preference pairs incorporating the SLM solutions and the
feedback. When trained with Supervised Fine-Tuning alone, Orca-Math achieves
81.50% on GSM8k pass@1 metric. With iterative preference learning, Orca-Math
achieves 86.81% pass@1. Orca-Math surpasses the performance of significantly
larger models such as LLAMA-2-70B, WizardMath-70B, Gemini-Pro, ChatGPT-3.5. It
also significantly outperforms other smaller models while using much smaller
data (hundreds of thousands vs. millions of problems).
Orca-Math, a 7-billion-parameter SLM based on Mistral-7B, achieves 86.81% on GSM8K without external tools, employing high-quality synthetic data and iterative preference learning.
synthetic dataset multi-agent setup iterative learning supervised fine-tuning iterative preference learning