Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Recent research has focused on enhancing the capability of smaller models
through imitation learning, drawing on the outputs generated by large
foundation models (LFMs). A number of issues impact the quality of these
models, ranging from limited imitation signals from shallow LFM outputs; small
scale homogeneous training data; and most notably a lack of rigorous evaluation
resulting in overestimating the small model's capability as they tend to learn
to imitate the style, but not the reasoning process of LFMs. To address these
challenges, we develop Orca (We are working with our legal team to publicly
release a diff of the model weights in accordance with LLaMA's release policy
to be published at https://aka.ms/orca-lm), a 13-billion parameter model that
learns to imitate the reasoning process of LFMs. Orca learns from rich signals
from GPT-4 including explanation traces; step-by-step thought processes; and
other complex instructions, guided by teacher assistance from ChatGPT. To
promote this progressive learning, we tap into large-scale and diverse
imitation data with judicious sampling and selection. Orca surpasses
conventional state-of-the-art instruction-tuned models such as Vicuna-13B by
more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard
(BBH) and 42% on AGIEval. Moreover, Orca reaches parity with ChatGPT on the BBH
benchmark and shows competitive performance (4 pts gap with optimized system
message) in professional and academic examinations like the SAT, LSAT, GRE, and
GMAT, both in zero-shot settings without CoT; while trailing behind GPT-4. Our
research indicates that learning from step-by-step explanations, whether these
are generated by humans or more advanced AI models, is a promising direction to
improve model capabilities and skills.
647e843218274bce0307a091
Subhabrata Mukherjee
extracted_confirmed
2023-06-06T01:20:13.175Z
5fd2dbbd9e5a0a03458e1fc8
/avatars/d0868aef6c180db8610184396443731a.svg
Subhabrata Mukherjee
user
subho
647e843218274bce0307a092
Arindam Mitra
claimed_verified
2023-06-06T17:23:01.086Z
609ee7bfa4d9efdf9ce888aa
/avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
Arindam Mitra
user
ari9dam
647e843218274bce0307a093
Ganesh Jawahar
647e843218274bce0307a094
Sahaj Agarwal
claimed_verified
2023-07-17T06:49:24.105Z
64b12642540a2ba57874349a
/avatars/e0ed6aa70e706865a6755c5fd10d5766.svg
Sahaj Agarwal
user
sahajgg
647e843218274bce0307a095
Hamid Palangi
647e843218274bce0307a096
Ahmed Awadallah
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Recent research has focused on enhancing the capability of smaller models
through imitation learning, drawing on the outputs generated by large
foundation models (LFMs). A number of issues impact the quality of these
models, ranging from limited imitation signals from shallow LFM outputs; small
scale homogeneous training data; and most notably a lack of rigorous evaluation
resulting in overestimating the small model's capability as they tend to learn
to imitate the style, but not the reasoning process of LFMs. To address these
challenges, we develop Orca (We are working with our legal team to publicly
release a diff of the model weights in accordance with LLaMA's release policy
to be published at https://aka.ms/orca-lm), a 13-billion parameter model that
learns to imitate the reasoning process of LFMs. Orca learns from rich signals
from GPT-4 including explanation traces; step-by-step thought processes; and
other complex instructions, guided by teacher assistance from ChatGPT. To
promote this progressive learning, we tap into large-scale and diverse
imitation data with judicious sampling and selection. Orca surpasses
conventional state-of-the-art instruction-tuned models such as Vicuna-13B by
more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard
(BBH) and 42% on AGIEval. Moreover, Orca reaches parity with ChatGPT on the BBH
benchmark and shows competitive performance (4 pts gap with optimized system
message) in professional and academic examinations like the SAT, LSAT, GRE, and
GMAT, both in zero-shot settings without CoT; while trailing behind GPT-4. Our
research indicates that learning from step-by-step explanations, whether these
are generated by humans or more advanced AI models, is a promising direction to
improve model capabilities and skills.
Orca, a 13-billion parameter model, enhances small models by imitating the reasoning process from large foundation models using rich signals and diverse data, surpassing existing models in complex reasoning benchmarks.
imitation learning large foundation models LFMs Orca GPT-4 explanation traces step-by-step thought processes complex instructions teacher assistance ChatGPT Big-Bench Hard (BBH) AGIEval SAT LSAT GRE GMAT zero-shot settings