title: Orca 2: Teaching Small Language Models How to Reason
summary: Orca 1 learns from rich signals, such as explanation traces, allowing it to
  outperform conventional instruction-tuned models on benchmarks like BigBench
  Hard and AGIEval. In Orca 2, we continue exploring how improved training
  signals can enhance smaller LMs' reasoning abilities. Research on training
  small LMs has often relied on imitation learning to replicate the output of
  more capable models. We contend that excessive emphasis on imitation may
  restrict the potential of smaller models. We seek to teach small LMs to employ
  different solution strategies for different tasks, potentially different from
  the one used by the larger model. For example, while larger models might
  provide a direct answer to a complex task, smaller models may not have the same
  capacity. In Orca 2, we teach the model various reasoning techniques
  (step-by-step, recall then generate, recall-reason-generate, direct answer,
  etc.). More crucially, we aim to help the model learn to determine the most
  effective solution strategy for each task. We evaluate Orca 2 using a
  comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
  tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
  similar size and attains performance levels similar or better to those of
  models 5-10x larger, as assessed on complex tasks that test advanced reasoning
  abilities in zero-shot settings. We open-source Orca 2 to encourage further
  research on the development, evaluation, and alignment of smaller LMs.
paper_authors._id: 655c0b866d02c2b1a92d1b09
paper_authors.name: Arindam Mitra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:07:48.109Z
paper_authors._id: 609ee7bfa4d9efdf9ce888aa
paper_authors.avatarUrl: /avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
paper_authors.fullname: Arindam Mitra
paper_authors.type: user
paper_authors.user: ari9dam
paper_authors._id: 655c0b866d02c2b1a92d1b0a
paper_authors.name: Luciano Del Corro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:07:55.253Z
paper_authors._id: 63b81aace60862785afd8ca2
paper_authors.avatarUrl: /avatars/f96ce78af6ad42514235bab811544789.svg
paper_authors.fullname: Luciano Del Corro
paper_authors.type: user
paper_authors.user: lucianodelcorro
paper_authors._id: 655c0b866d02c2b1a92d1b0b
paper_authors.name: Shweti Mahajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:08:01.973Z
paper_authors._id: 6553dc5d13df3a4e4ec34a4f
paper_authors.avatarUrl: /avatars/1059efc2c5bc3058c2b70a84c41f26a9.svg
paper_authors.fullname: Shweti Mahajan
paper_authors.type: user
paper_authors.user: shmahaj
paper_authors._id: 655c0b866d02c2b1a92d1b0c
paper_authors.name: Andres Codas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:08:08.259Z
paper_authors._id: 64501ad24f731658826a0bbe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pqdKbO1j0ilAwKcJz1rph.jpeg
paper_authors.fullname: Andres Codas
paper_authors.type: user
paper_authors.user: andrescodas
paper_authors._id: 655c0b866d02c2b1a92d1b0d
paper_authors.name: Clarisse Simoes
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:08:26.990Z
paper_authors._id: 61a563bfd048de7ad384b7b7
paper_authors.avatarUrl: /avatars/84e0f1af7c37aa6db707437b9ffc7f80.svg
paper_authors.fullname: Clarisse Simoes Ribeiro
paper_authors.type: user
paper_authors.user: clasimoes
paper_authors._id: 655c0b866d02c2b1a92d1b0e
paper_authors.name: Sahaj Agrawal
paper_authors._id: 655c0b866d02c2b1a92d1b0f
paper_authors.name: Xuxi Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:08:55.971Z
paper_authors._id: 614ae3bb51b6682b9e56fd8d
paper_authors.avatarUrl: /avatars/306f6fce597883ac85bce12d0669b3ce.svg
paper_authors.fullname: Xuxi Chen
paper_authors.type: user
paper_authors.user: Xuxi
paper_authors._id: 655c0b866d02c2b1a92d1b10
paper_authors.name: Anastasia Razdaibiedina
paper_authors._id: 655c0b866d02c2b1a92d1b11
paper_authors.name: Erik Jones
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-03T08:56:34.436Z
paper_authors._id: 63518b69d95a8aa0c2247488
paper_authors.avatarUrl: /avatars/8ab0cb1e4e5d66c0018e22df035a6a9e.svg
paper_authors.fullname: Erik Jones
paper_authors.type: user
paper_authors.user: erikjones313
paper_authors._id: 655c0b866d02c2b1a92d1b12
paper_authors.name: Kriti Aggarwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-21T10:24:28.944Z
paper_authors._id: 655c25ab0555d96ad977ade7
paper_authors.avatarUrl: /avatars/8c6f23cd9f97dff15f8b1ed1d5e7c537.svg
paper_authors.fullname: Kriti Aggarwal
paper_authors.type: user
paper_authors.user: kritiaggarwal
paper_authors._id: 655c0b866d02c2b1a92d1b13
paper_authors.name: Hamid Palangi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:09:27.486Z
paper_authors._id: 6189ddd0992df2640e3e7d40
paper_authors.avatarUrl: /avatars/925f2308fc412ae352b57a1b71815028.svg
paper_authors.fullname: Hamid Palangi
paper_authors.type: user
paper_authors.user: hamidpalangi
paper_authors._id: 655c0b866d02c2b1a92d1b14
paper_authors.name: Guoqing Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:09:37.596Z
paper_authors._id: 6552620be170f85075ed5d24
paper_authors.avatarUrl: /avatars/cc1c783b2a6a7efe25dd01420993a1be.svg
paper_authors.fullname: Guoqing Zheng
paper_authors.type: user
paper_authors.user: zhenggq
paper_authors._id: 655c0b866d02c2b1a92d1b15
paper_authors.name: Corby Rosset
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:09:44.010Z
paper_authors._id: 64ec43147e2ec711a761c594
paper_authors.avatarUrl: /avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
paper_authors.fullname: corby
paper_authors.type: user
paper_authors.user: corbyrosset
paper_authors._id: 655c0b866d02c2b1a92d1b16
paper_authors.name: Hamed Khanpour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:09:50.929Z
paper_authors._id: 6553d3d55ba75868f6651023
paper_authors.avatarUrl: /avatars/c052a841e13fb7978905f5c092fbc685.svg
paper_authors.fullname: Hamed Khanpour
paper_authors.type: user
paper_authors.user: hamedkhanpour
paper_authors._id: 655c0b866d02c2b1a92d1b17
paper_authors.name: Ahmed Awadallah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T13:10:25.376Z
paper_authors._id: 6557a41ee74d1065f7d05c51
paper_authors.avatarUrl: /avatars/b29e57fc61d1d3012637f84b82c14818.svg
paper_authors.fullname: Ahmed Awadallah
paper_authors.type: user
paper_authors.user: AhmedAwadallah
paper_title: Orca 2: Teaching Small Language Models How to Reason
paper_summary: Orca 1 learns from rich signals, such as explanation traces, allowing it to
  outperform conventional instruction-tuned models on benchmarks like BigBench
  Hard and AGIEval. In Orca 2, we continue exploring how improved training
  signals can enhance smaller LMs' reasoning abilities. Research on training
  small LMs has often relied on imitation learning to replicate the output of
  more capable models. We contend that excessive emphasis on imitation may
  restrict the potential of smaller models. We seek to teach small LMs to employ
  different solution strategies for different tasks, potentially different from
  the one used by the larger model. For example, while larger models might
  provide a direct answer to a complex task, smaller models may not have the same
  capacity. In Orca 2, we teach the model various reasoning techniques
  (step-by-step, recall then generate, recall-reason-generate, direct answer,
  etc.). More crucially, we aim to help the model learn to determine the most
  effective solution strategy for each task. We evaluate Orca 2 using a
  comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
  tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
  similar size and attains performance levels similar or better to those of
  models 5-10x larger, as assessed on complex tasks that test advanced reasoning
  abilities in zero-shot settings. We open-source Orca 2 to encourage further
  research on the development, evaluation, and alignment of smaller LMs.
paper_ai_keywords: imitation learning solution strategies step-by-step recall then generate recall-reason-generate direct answer reasoning techniques zero-shot settings alignment