Orca 2: Teaching Small Language Models How to Reason
Orca 1 learns from rich signals, such as explanation traces, allowing it to
outperform conventional instruction-tuned models on benchmarks like BigBench
Hard and AGIEval. In Orca 2, we continue exploring how improved training
signals can enhance smaller LMs' reasoning abilities. Research on training
small LMs has often relied on imitation learning to replicate the output of
more capable models. We contend that excessive emphasis on imitation may
restrict the potential of smaller models. We seek to teach small LMs to employ
different solution strategies for different tasks, potentially different from
the one used by the larger model. For example, while larger models might
provide a direct answer to a complex task, smaller models may not have the same
capacity. In Orca 2, we teach the model various reasoning techniques
(step-by-step, recall then generate, recall-reason-generate, direct answer,
etc.). More crucially, we aim to help the model learn to determine the most
effective solution strategy for each task. We evaluate Orca 2 using a
comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
similar size and attains performance levels similar or better to those of
models 5-10x larger, as assessed on complex tasks that test advanced reasoning
abilities in zero-shot settings. We open-source Orca 2 to encourage further
research on the development, evaluation, and alignment of smaller LMs.
655c0b866d02c2b1a92d1b09
Arindam Mitra
admin_assigned
2023-11-21T13:07:48.109Z
609ee7bfa4d9efdf9ce888aa
/avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
Arindam Mitra
user
ari9dam
655c0b866d02c2b1a92d1b0a
Luciano Del Corro
admin_assigned
2023-11-21T13:07:55.253Z
63b81aace60862785afd8ca2
/avatars/f96ce78af6ad42514235bab811544789.svg
Luciano Del Corro
user
lucianodelcorro
655c0b866d02c2b1a92d1b0b
Shweti Mahajan
admin_assigned
2023-11-21T13:08:01.973Z
6553dc5d13df3a4e4ec34a4f
/avatars/1059efc2c5bc3058c2b70a84c41f26a9.svg
Shweti Mahajan
user
shmahaj
655c0b866d02c2b1a92d1b0c
Andres Codas
admin_assigned
2023-11-21T13:08:08.259Z
64501ad24f731658826a0bbe
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pqdKbO1j0ilAwKcJz1rph.jpeg
Andres Codas
user
andrescodas
655c0b866d02c2b1a92d1b0d
Clarisse Simoes
admin_assigned
2023-11-21T13:08:26.990Z
61a563bfd048de7ad384b7b7
/avatars/84e0f1af7c37aa6db707437b9ffc7f80.svg
Clarisse Simoes Ribeiro
user
clasimoes
655c0b866d02c2b1a92d1b0e
Sahaj Agrawal
655c0b866d02c2b1a92d1b0f
Xuxi Chen
admin_assigned
2023-11-21T13:08:55.971Z
614ae3bb51b6682b9e56fd8d
/avatars/306f6fce597883ac85bce12d0669b3ce.svg
Xuxi Chen
user
Xuxi
655c0b866d02c2b1a92d1b10
Anastasia Razdaibiedina
655c0b866d02c2b1a92d1b11
Erik Jones
claimed_verified
2024-04-03T08:56:34.436Z
63518b69d95a8aa0c2247488
/avatars/8ab0cb1e4e5d66c0018e22df035a6a9e.svg
Erik Jones
user
erikjones313
655c0b866d02c2b1a92d1b12
Kriti Aggarwal
claimed_verified
2023-11-21T10:24:28.944Z
655c25ab0555d96ad977ade7
/avatars/8c6f23cd9f97dff15f8b1ed1d5e7c537.svg
Kriti Aggarwal
user
kritiaggarwal
655c0b866d02c2b1a92d1b13
Hamid Palangi
admin_assigned
2023-11-21T13:09:27.486Z
6189ddd0992df2640e3e7d40
/avatars/925f2308fc412ae352b57a1b71815028.svg
Hamid Palangi
user
hamidpalangi
655c0b866d02c2b1a92d1b14
Guoqing Zheng
admin_assigned
2023-11-21T13:09:37.596Z
6552620be170f85075ed5d24
/avatars/cc1c783b2a6a7efe25dd01420993a1be.svg
Guoqing Zheng
user
zhenggq
655c0b866d02c2b1a92d1b15
Corby Rosset
admin_assigned
2023-11-21T13:09:44.010Z
64ec43147e2ec711a761c594
/avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
corby
user
corbyrosset
655c0b866d02c2b1a92d1b16
Hamed Khanpour
admin_assigned
2023-11-21T13:09:50.929Z
6553d3d55ba75868f6651023
/avatars/c052a841e13fb7978905f5c092fbc685.svg
Hamed Khanpour
user
hamedkhanpour
655c0b866d02c2b1a92d1b17
Ahmed Awadallah
admin_assigned
2023-11-21T13:10:25.376Z
6557a41ee74d1065f7d05c51
/avatars/b29e57fc61d1d3012637f84b82c14818.svg
Ahmed Awadallah
user
AhmedAwadallah
Orca 2: Teaching Small Language Models How to Reason
Orca 1 learns from rich signals, such as explanation traces, allowing it to
outperform conventional instruction-tuned models on benchmarks like BigBench
Hard and AGIEval. In Orca 2, we continue exploring how improved training
signals can enhance smaller LMs' reasoning abilities. Research on training
small LMs has often relied on imitation learning to replicate the output of
more capable models. We contend that excessive emphasis on imitation may
restrict the potential of smaller models. We seek to teach small LMs to employ
different solution strategies for different tasks, potentially different from
the one used by the larger model. For example, while larger models might
provide a direct answer to a complex task, smaller models may not have the same
capacity. In Orca 2, we teach the model various reasoning techniques
(step-by-step, recall then generate, recall-reason-generate, direct answer,
etc.). More crucially, we aim to help the model learn to determine the most
effective solution strategy for each task. We evaluate Orca 2 using a
comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
similar size and attains performance levels similar or better to those of
models 5-10x larger, as assessed on complex tasks that test advanced reasoning
abilities in zero-shot settings. We open-source Orca 2 to encourage further
research on the development, evaluation, and alignment of smaller LMs.
Orca 2 enhances smaller language models' reasoning abilities by teaching them diverse solution strategies, outperforming larger models on complex reasoning tasks.
imitation learning reasoning techniques step-by-step recall then generate recall-reason-generate direct answer zero-shot settings