title: Better Alignment with Instruction Back-and-Forth Translation
summary: We propose a new method, instruction back-and-forth translation, to construct
  high-quality synthetic data grounded in world knowledge for aligning large
  language models (LLMs). Given documents from a web corpus, we generate and
  curate synthetic instructions using the backtranslation approach proposed by Li
  et al.(2023a), and rewrite the responses to improve their quality further based
  on the initial documents. Fine-tuning with the resulting (backtranslated
  instruction, rewritten response) pairs yields higher win rates on AlpacaEval
  than using other common instruction datasets such as Humpback, ShareGPT, Open
  Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the
  responses with an LLM outperforms direct distillation, and the two generated
  text distributions exhibit significant distinction in embedding space. Further
  analysis shows that our backtranslated instructions are of higher quality than
  other sources of synthetic instructions, while our responses are more diverse
  and complex than those obtained from distillation. Overall we find that
  instruction back-and-forth translation combines the best of both worlds --
  making use of the information diversity and quantity found on the web, while
  ensuring the quality of the responses which is necessary for effective
  alignment.
paper_authors._id: 66b58c938533192be928639b
paper_authors.name: Thao Nguyen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:47:27.838Z
paper_authors._id: 64ba4923921dbcbc96006fb8
paper_authors.avatarUrl: /avatars/4eab3991e8502f72ddf8ca4825c2ad65.svg
paper_authors.fullname: Thao Nguyen
paper_authors.type: user
paper_authors.user: thaottn
paper_authors._id: 66b58c938533192be928639c
paper_authors.name: Jeffrey Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-09T10:10:05.779Z
paper_authors._id: 63ca51a82ffd350b485f9fa5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674203513156-noauth.jpeg
paper_authors.fullname: Jeffrey Li
paper_authors.type: user
paper_authors.user: JeffreyLi
paper_authors._id: 66b58c938533192be928639d
paper_authors.name: Sewoong Oh
paper_authors._id: 66b58c938533192be928639e
paper_authors.name: Ludwig Schmidt
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-09T10:10:26.914Z
paper_authors._id: 6252375c818a5dc29ab9d945
paper_authors.avatarUrl: /avatars/cd4f0b220ea36e13ff71157cffe9d9d1.svg
paper_authors.fullname: Ludwig Schmidt
paper_authors.type: user
paper_authors.user: ludwigschmidt
paper_authors._id: 66b58c938533192be928639f
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-09T10:10:35.766Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 66b58c938533192be92863a0
paper_authors.name: Luke Zettlemoyer
paper_authors._id: 66b58c938533192be92863a1
paper_authors.name: Xian Li
paper_title: Better Alignment with Instruction Back-and-Forth Translation
paper_summary: We propose a new method, instruction back-and-forth translation, to construct
  high-quality synthetic data grounded in world knowledge for aligning large
  language models (LLMs). Given documents from a web corpus, we generate and
  curate synthetic instructions using the backtranslation approach proposed by Li
  et al.(2023a), and rewrite the responses to improve their quality further based
  on the initial documents. Fine-tuning with the resulting (backtranslated
  instruction, rewritten response) pairs yields higher win rates on AlpacaEval
  than using other common instruction datasets such as Humpback, ShareGPT, Open
  Orca, Alpaca-GPT4 and Self-instruct. We also demonstrate that rewriting the
  responses with an LLM outperforms direct distillation, and the two generated
  text distributions exhibit significant distinction in embedding space. Further
  analysis shows that our backtranslated instructions are of higher quality than
  other sources of synthetic instructions, while our responses are more diverse
  and complex than those obtained from distillation. Overall we find that
  instruction back-and-forth translation combines the best of both worlds --
  making use of the information diversity and quantity found on the web, while
  ensuring the quality of the responses which is necessary for effective
  alignment.
paper_ai_keywords: instruction back-and-forth translation fine-tuning large language models (LLMs) backtranslation synthetic instructions rewritten responses AlpacaEval Humpback ShareGPT Orca Alpaca-GPT4 Self-instruct LLM direct distillation embedding space diverse text distributions quality of instructions diversity of responses comprehensive range of multimodal tasks multimodal alignment