title: AgentInstruct: Toward Generative Teaching with Agentic Flows
summary: Synthetic data is becoming increasingly important for accelerating the
  development of language models, both large and small. Despite several
  successful use cases, researchers also raised concerns around model collapse
  and drawbacks of imitating other models. This discrepancy can be attributed to
  the fact that synthetic data varies in quality and diversity. Effective use of
  synthetic data usually requires significant human effort in curating the data.
  We focus on using synthetic data for post-training, specifically creating data
  by powerful models to teach a new skill or behavior to another model, we refer
  to this setting as Generative Teaching. We introduce AgentInstruct, an
  extensible agentic framework for automatically creating large amounts of
  diverse and high-quality synthetic data. AgentInstruct can create both the
  prompts and responses, using only raw data sources like text documents and code
  files as seeds. We demonstrate the utility of AgentInstruct by creating a post
  training dataset of 25M pairs to teach language models different skills, such
  as text editing, creative writing, tool usage, coding, reading comprehension,
  etc. The dataset can be used for instruction tuning of any base model. We
  post-train Mistral-7b with the data. When comparing the resulting model Orca-3
  to Mistral-7b-Instruct (which uses the same base model), we observe significant
  improvements across many benchmarks. For example, 40% improvement on AGIEval,
  19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and
  45% improvement on AlpacaEval. Additionally, it consistently outperforms other
  models such as LLAMA-8B-instruct and GPT-3.5-turbo.
paper_authors._id: 668cdf6877108dfc2e882cf4
paper_authors.name: Arindam Mitra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:35:42.409Z
paper_authors._id: 609ee7bfa4d9efdf9ce888aa
paper_authors.avatarUrl: /avatars/b5429d85c030a9f08e1d2bb37ce9974d.svg
paper_authors.fullname: Arindam Mitra
paper_authors.type: user
paper_authors.user: ari9dam
paper_authors._id: 668cdf6877108dfc2e882cf5
paper_authors.name: Luciano Del Corro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:35:48.096Z
paper_authors._id: 63b81aace60862785afd8ca2
paper_authors.avatarUrl: /avatars/f96ce78af6ad42514235bab811544789.svg
paper_authors.fullname: Luciano Del Corro
paper_authors.type: user
paper_authors.user: lucianodelcorro
paper_authors._id: 668cdf6877108dfc2e882cf6
paper_authors.name: Guoqing Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:36:55.589Z
paper_authors._id: 6552620be170f85075ed5d24
paper_authors.avatarUrl: /avatars/cc1c783b2a6a7efe25dd01420993a1be.svg
paper_authors.fullname: Guoqing Zheng
paper_authors.type: user
paper_authors.user: zhenggq
paper_authors._id: 668cdf6877108dfc2e882cf7
paper_authors.name: Shweti Mahajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:37:02.059Z
paper_authors._id: 6553dc5d13df3a4e4ec34a4f
paper_authors.avatarUrl: /avatars/1059efc2c5bc3058c2b70a84c41f26a9.svg
paper_authors.fullname: Shweti Mahajan
paper_authors.type: user
paper_authors.user: shmahaj
paper_authors._id: 668cdf6877108dfc2e882cf8
paper_authors.name: Dany Rouhana
paper_authors._id: 668cdf6877108dfc2e882cf9
paper_authors.name: Andres Codas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:37:14.460Z
paper_authors._id: 64501ad24f731658826a0bbe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pqdKbO1j0ilAwKcJz1rph.jpeg
paper_authors.fullname: Andres Codas
paper_authors.type: user
paper_authors.user: andrescodas
paper_authors._id: 668cdf6877108dfc2e882cfa
paper_authors.name: Yadong Lu
paper_authors._id: 668cdf6877108dfc2e882cfb
paper_authors.name: Wei-ge Chen
paper_authors._id: 668cdf6877108dfc2e882cfc
paper_authors.name: Olga Vrousgos
paper_authors._id: 668cdf6877108dfc2e882cfd
paper_authors.name: Corby Rosset
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:38:18.747Z
paper_authors._id: 64ec43147e2ec711a761c594
paper_authors.avatarUrl: /avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
paper_authors.fullname: corby
paper_authors.type: user
paper_authors.user: corbyrosset
paper_authors._id: 668cdf6877108dfc2e882cfe
paper_authors.name: Fillipe Silva
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-03T07:49:57.612Z
paper_authors._id: 64091f8e3461c51cf7372c64
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678319404440-noauth.jpeg
paper_authors.fullname: Santos
paper_authors.type: user
paper_authors.user: Fillipe
paper_authors._id: 668cdf6877108dfc2e882cff
paper_authors.name: Hamed Khanpour
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:38:33.890Z
paper_authors._id: 6553d3d55ba75868f6651023
paper_authors.avatarUrl: /avatars/c052a841e13fb7978905f5c092fbc685.svg
paper_authors.fullname: Hamed Khanpour
paper_authors.type: user
paper_authors.user: hamedkhanpour
paper_authors._id: 668cdf6877108dfc2e882d00
paper_authors.name: Yash Lara
paper_authors._id: 668cdf6877108dfc2e882d01
paper_authors.name: Ahmed Awadallah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:38:54.056Z
paper_authors._id: 6557a41ee74d1065f7d05c51
paper_authors.avatarUrl: /avatars/b29e57fc61d1d3012637f84b82c14818.svg
paper_authors.fullname: Ahmed Awadallah
paper_authors.type: user
paper_authors.user: AhmedAwadallah
paper_title: AgentInstruct: Toward Generative Teaching with Agentic Flows
paper_summary: Synthetic data is becoming increasingly important for accelerating the
  development of language models, both large and small. Despite several
  successful use cases, researchers also raised concerns around model collapse
  and drawbacks of imitating other models. This discrepancy can be attributed to
  the fact that synthetic data varies in quality and diversity. Effective use of
  synthetic data usually requires significant human effort in curating the data.
  We focus on using synthetic data for post-training, specifically creating data
  by powerful models to teach a new skill or behavior to another model, we refer
  to this setting as Generative Teaching. We introduce AgentInstruct, an
  extensible agentic framework for automatically creating large amounts of
  diverse and high-quality synthetic data. AgentInstruct can create both the
  prompts and responses, using only raw data sources like text documents and code
  files as seeds. We demonstrate the utility of AgentInstruct by creating a post
  training dataset of 25M pairs to teach language models different skills, such
  as text editing, creative writing, tool usage, coding, reading comprehension,
  etc. The dataset can be used for instruction tuning of any base model. We
  post-train Mistral-7b with the data. When comparing the resulting model Orca-3
  to Mistral-7b-Instruct (which uses the same base model), we observe significant
  improvements across many benchmarks. For example, 40% improvement on AGIEval,
  19% improvement on MMLU, 54% improvement on GSM8K, 38% improvement on BBH and
  45% improvement on AlpacaEval. Additionally, it consistently outperforms other
  models such as LLAMA-8B-instruct and GPT-3.5-turbo.
paper_ai_summary: AgentInstruct, an agentic framework, generates high-quality synthetic data for post-training language models, leading to significant performance improvements across various benchmarks.
paper_ai_keywords: AgentInstruct generative teaching language models post-training prompts responses instruction tuning Mistral-7b Orca-3 Mistral-7b-Instruct AGIEval MMLU GSM8K BBH AlpacaEval LLAMA-8B-instruct GPT-3.5-turbo