AgentInstruct: Toward Generative Teaching with Agentic Flows
Synthetic data is becoming increasingly important for accelerating the
development of language models, both large and small. Despite several
successful use cases, researchers also raised concerns around model collapse
and drawbacks of imitating other models. This discrepancy can be attributed to
the fact that synthetic data varies in quality and diversity. Effective use of
synthetic data usually requires significant human effort in curating the data.
We focus on using synthetic data for post-training, specifically creating data
by powerful models to teach a new skill or behavior to another model, we refer
to this setting as Generative Teaching. We introduce AgentInstruct, an
extensible agentic framework for automatically creating large amounts of
diverse and high-quality synthetic data. AgentInstruct can create both the
prompts and responses, using only raw data sources like text documents and code
files as seeds. We demonstrate the utility of AgentInstruct by creating a post
training dataset of 25M pairs to teach language models different skills, such
as text editing, creative writing, tool usage, coding, reading comprehension,
etc. The dataset can be used for instruction tuning of any base model. We
post-train Mistral-7b with the data. When comparing the resulting model Orca-3
to Mistral-7b-Instruct (which uses the same base model), we observe significant
improvements across many benchmarks. For example, 40% improvement on AGIEval,
19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and
45% improvement on AlpacaEval. Additionally, it consistently outperforms other
models such as LLAMA-8B-instruct and GPT-3.5-turbo.
668cdf6877108dfc2e882cf4
Arindam Mitra
admin_assigned
2024-07-10T10:35:42.409Z
609ee7bfa4d9efdf9ce888aa
/avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
Arindam Mitra
user
ari9dam
668cdf6877108dfc2e882cf5
Luciano Del Corro
admin_assigned
2024-07-10T10:35:48.096Z
63b81aace60862785afd8ca2
/avatars/f96ce78af6ad42514235bab811544789.svg
Luciano Del Corro
user
lucianodelcorro
668cdf6877108dfc2e882cf6
Guoqing Zheng
admin_assigned
2024-07-10T10:36:55.589Z
6552620be170f85075ed5d24
/avatars/cc1c783b2a6a7efe25dd01420993a1be.svg
Guoqing Zheng
user
zhenggq
668cdf6877108dfc2e882cf7
Shweti Mahajan
admin_assigned
2024-07-10T10:37:02.059Z
6553dc5d13df3a4e4ec34a4f
/avatars/1059efc2c5bc3058c2b70a84c41f26a9.svg
Shweti Mahajan
user
shmahaj
668cdf6877108dfc2e882cf8
Dany Rouhana
668cdf6877108dfc2e882cf9
Andres Codas
admin_assigned
2024-07-10T10:37:14.460Z
64501ad24f731658826a0bbe
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pqdKbO1j0ilAwKcJz1rph.jpeg
Andres Codas
user
andrescodas
668cdf6877108dfc2e882cfa
Yadong Lu
668cdf6877108dfc2e882cfb
Wei-ge Chen
668cdf6877108dfc2e882cfc
Olga Vrousgos
668cdf6877108dfc2e882cfd
Corby Rosset
admin_assigned
2024-07-10T10:38:18.747Z
64ec43147e2ec711a761c594
/avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
corby
user
corbyrosset
668cdf6877108dfc2e882cfe
Fillipe Silva
claimed_verified
2024-09-03T07:49:57.612Z
64091f8e3461c51cf7372c64
https://cdn-avatars.huggingface.co/v1/production/uploads/1678319404440-noauth.jpeg
Santos
user
Fillipe
668cdf6877108dfc2e882cff
Hamed Khanpour
admin_assigned
2024-07-10T10:38:33.890Z
6553d3d55ba75868f6651023
/avatars/c052a841e13fb7978905f5c092fbc685.svg
Hamed Khanpour
user
hamedkhanpour
668cdf6877108dfc2e882d00
Yash Lara
668cdf6877108dfc2e882d01
Ahmed Awadallah
admin_assigned
2024-07-10T10:38:54.056Z
6557a41ee74d1065f7d05c51
/avatars/b29e57fc61d1d3012637f84b82c14818.svg
Ahmed Awadallah
user
AhmedAwadallah
AgentInstruct: Toward Generative Teaching with Agentic Flows
Synthetic data is becoming increasingly important for accelerating the
development of language models, both large and small. Despite several
successful use cases, researchers also raised concerns around model collapse
and drawbacks of imitating other models. This discrepancy can be attributed to
the fact that synthetic data varies in quality and diversity. Effective use of
synthetic data usually requires significant human effort in curating the data.
We focus on using synthetic data for post-training, specifically creating data
by powerful models to teach a new skill or behavior to another model, we refer
to this setting as Generative Teaching. We introduce AgentInstruct, an
extensible agentic framework for automatically creating large amounts of
diverse and high-quality synthetic data. AgentInstruct can create both the
prompts and responses, using only raw data sources like text documents and code
files as seeds. We demonstrate the utility of AgentInstruct by creating a post
training dataset of 25M pairs to teach language models different skills, such
as text editing, creative writing, tool usage, coding, reading comprehension,
etc. The dataset can be used for instruction tuning of any base model. We
post-train Mistral-7b with the data. When comparing the resulting model Orca-3
to Mistral-7b-Instruct (which uses the same base model), we observe significant
improvements across many benchmarks. For example, 40% improvement on AGIEval,
19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and
45% improvement on AlpacaEval. Additionally, it consistently outperforms other
models such as LLAMA-8B-instruct and GPT-3.5-turbo.
AgentInstruct, an agentic framework, generates high-quality synthetic data for post-training language models, leading to significant performance improvements across various benchmarks.
AgentInstruct generative teaching language models post-training prompts responses instruction tuning Mistral-7b Orca-3 Mistral-7b-Instruct AGIEval MMLU GSM8K BBH AlpacaEval LLAMA-8B-instruct GPT-3.5-turbo