title: Self-Alignment with Instruction Backtranslation
summary: We present a scalable method to build a high quality instruction following
  language model by automatically labelling human-written text with corresponding
  instructions. Our approach, named instruction backtranslation, starts with a
  language model finetuned on a small amount of seed data, and a given web
  corpus. The seed model is used to construct training examples by generating
  instruction prompts for web documents (self-augmentation), and then selecting
  high quality examples from among these candidates (self-curation). This data is
  then used to finetune a stronger model. Finetuning LLaMa on two iterations of
  our approach yields a model that outperforms all other LLaMa-based models on
  the Alpaca leaderboard not relying on distillation data, demonstrating highly
  effective self-alignment.
paper_authors._id: 64d9acf1089bc502cef3610f
paper_authors.name: Xian Li
paper_authors._id: 64d9acf1089bc502cef36110
paper_authors.name: Ping Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:48:33.632Z
paper_authors._id: 646c004e31968a60a022590e
paper_authors.avatarUrl: /avatars/349e2ed4feef1be050fd9903dccec1e1.svg
paper_authors.fullname: Ping Yu
paper_authors.type: user
paper_authors.user: pingyu
paper_authors._id: 64d9acf1089bc502cef36111
paper_authors.name: Chunting Zhou
paper_authors._id: 64d9acf1089bc502cef36112
paper_authors.name: Timo Schick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:49:02.082Z
paper_authors._id: 613710606ecbbda5c86755c8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1636366232124-613710606ecbbda5c86755c8.png
paper_authors.fullname: Timo Schick
paper_authors.type: user
paper_authors.user: timoschick
paper_authors._id: 64d9acf1089bc502cef36113
paper_authors.name: Luke Zettlemoyer
paper_authors._id: 64d9acf1089bc502cef36114
paper_authors.name: Omer Levy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:49:29.778Z
paper_authors._id: 610b71d7f5dd7bcefedaff55
paper_authors.avatarUrl: /avatars/b489c021e906010e661313da0b4aafab.svg
paper_authors.fullname: Omer Levy
paper_authors.type: user
paper_authors.user: omerlevy
paper_authors._id: 64d9acf1089bc502cef36115
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:49:50.913Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 64d9acf1089bc502cef36116
paper_authors.name: Mike Lewis
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-15T00:50:09.776Z
paper_authors._id: 5e67eda71009063689407484
paper_authors.avatarUrl: /avatars/6d7688faba888d463584eaa02ad011e3.svg
paper_authors.fullname: Mike Lewis
paper_authors.type: user
paper_authors.user: mikelewis0
paper_title: Self-Alignment with Instruction Backtranslation
paper_summary: We present a scalable method to build a high quality instruction following
  language model by automatically labelling human-written text with corresponding
  instructions. Our approach, named instruction backtranslation, starts with a
  language model finetuned on a small amount of seed data, and a given web
  corpus. The seed model is used to construct training examples by generating
  instruction prompts for web documents (self-augmentation), and then selecting
  high quality examples from among these candidates (self-curation). This data is
  then used to finetune a stronger model. Finetuning LLaMa on two iterations of
  our approach yields a model that outperforms all other LLaMa-based models on
  the Alpaca leaderboard not relying on distillation data, demonstrating highly
  effective self-alignment.
paper_ai_summary: A scalable instruction-following language model is built using auto-labelling and iterative self-augmentation and self-curation, outperforming other LLaMa-based models on Alpaca.
paper_ai_keywords: instruction backtranslation language model finetuned seed data web corpus instruction prompts self-augmentation self-curation LLaMa Alpaca leaderboard self-alignment