title: Orca-Math: Unlocking the potential of SLMs in Grade School Math
summary: Mathematical word problem-solving has long been recognized as a complex task
  for small language models (SLMs). A recent study hypothesized that the smallest
  model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34
  billion parameters. To reach this level of performance with smaller models,
  researcher often train SLMs to generate Python code or use tools to help avoid
  calculation errors. Additionally, they employ ensembling, where outputs of up
  to 100 model runs are combined to arrive at a more accurate result. Result
  selection is done using consensus, majority vote or a separate a verifier model
  used in conjunction with the SLM. Ensembling provides a substantial boost in
  accuracy but at a significant cost increase with multiple calls to the model
  (e.g., Phi-GSM uses top-48 to boost the performance from 68.2 to 81.5).
    In this work, we present Orca-Math, a 7-billion-parameter SLM based on the
  Mistral-7B, which achieves 86.81% on GSM8k without the need for multiple model
  calls or the use of verifiers, code execution or any other external tools. Our
  approach has the following key elements: (1) A high quality synthetic dataset
  of 200K math problems created using a multi-agent setup where agents
  collaborate to create the data, (2) An iterative learning techniques that
  enables the SLM to practice solving problems, receive feedback on its solutions
  and learn from preference pairs incorporating the SLM solutions and the
  feedback. When trained with Supervised Fine-Tuning alone, Orca-Math achieves
  81.50% on GSM8k pass@1 metric. With iterative preference learning, Orca-Math
  achieves 86.81% pass@1. Orca-Math surpasses the performance of significantly
  larger models such as LLAMA-2-70B, WizardMath-70B, Gemini-Pro, ChatGPT-3.5. It
  also significantly outperforms other smaller models while using much smaller
  data (hundreds of thousands vs. millions of problems).
paper_authors._id: 65dbf639139bc4eee34a10ff
paper_authors.name: Arindam Mitra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T10:19:55.384Z
paper_authors._id: 609ee7bfa4d9efdf9ce888aa
paper_authors.avatarUrl: /avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
paper_authors.fullname: Arindam Mitra
paper_authors.type: user
paper_authors.user: ari9dam
paper_authors._id: 65dbf639139bc4eee34a1100
paper_authors.name: Hamed Khanpour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T10:20:01.313Z
paper_authors._id: 6553d3d55ba75868f6651023
paper_authors.avatarUrl: /avatars/c052a841e13fb7978905f5c092fbc685.svg
paper_authors.fullname: Hamed Khanpour
paper_authors.type: user
paper_authors.user: hamedkhanpour
paper_authors._id: 65dbf639139bc4eee34a1101
paper_authors.name: Corby Rosset
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T10:20:07.586Z
paper_authors._id: 64ec43147e2ec711a761c594
paper_authors.avatarUrl: /avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
paper_authors.fullname: corby
paper_authors.type: user
paper_authors.user: corbyrosset
paper_authors._id: 65dbf639139bc4eee34a1102
paper_authors.name: Ahmed Awadallah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T10:20:19.898Z
paper_authors._id: 6557a41ee74d1065f7d05c51
paper_authors.avatarUrl: /avatars/b29e57fc61d1d3012637f84b82c14818.svg
paper_authors.fullname: Ahmed Awadallah
paper_authors.type: user
paper_authors.user: AhmedAwadallah
paper_title: Orca-Math: Unlocking the potential of SLMs in Grade School Math
paper_summary: Mathematical word problem-solving has long been recognized as a complex task
  for small language models (SLMs). A recent study hypothesized that the smallest
  model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34
  billion parameters. To reach this level of performance with smaller models,
  researcher often train SLMs to generate Python code or use tools to help avoid
  calculation errors. Additionally, they employ ensembling, where outputs of up
  to 100 model runs are combined to arrive at a more accurate result. Result
  selection is done using consensus, majority vote or a separate a verifier model
  used in conjunction with the SLM. Ensembling provides a substantial boost in
  accuracy but at a significant cost increase with multiple calls to the model
  (e.g., Phi-GSM uses top-48 to boost the performance from 68.2 to 81.5).
    In this work, we present Orca-Math, a 7-billion-parameter SLM based on the
  Mistral-7B, which achieves 86.81% on GSM8k without the need for multiple model
  calls or the use of verifiers, code execution or any other external tools. Our
  approach has the following key elements: (1) A high quality synthetic dataset
  of 200K math problems created using a multi-agent setup where agents
  collaborate to create the data, (2) An iterative learning techniques that
  enables the SLM to practice solving problems, receive feedback on its solutions
  and learn from preference pairs incorporating the SLM solutions and the
  feedback. When trained with Supervised Fine-Tuning alone, Orca-Math achieves
  81.50% on GSM8k pass@1 metric. With iterative preference learning, Orca-Math
  achieves 86.81% pass@1. Orca-Math surpasses the performance of significantly
  larger models such as LLAMA-2-70B, WizardMath-70B, Gemini-Pro, ChatGPT-3.5. It
  also significantly outperforms other smaller models while using much smaller
  data (hundreds of thousands vs. millions of problems).
paper_ai_summary: Orca-Math, a 7-billion-parameter SLM based on Mistral-7B, achieves 86.81% on GSM8K without external tools, employing high-quality synthetic data and iterative preference learning.
paper_ai_keywords: synthetic dataset multi-agent setup iterative learning supervised fine-tuning iterative preference learning