title: Constraint Back-translation Improves Complex Instruction Following of
    Large Language Models
summary: Large language models (LLMs) struggle to follow instructions with complex
  constraints in format, length, etc. Following the conventional
  instruction-tuning practice, previous works conduct post-training on complex
  instruction-response pairs generated by feeding complex instructions to
  advanced LLMs. However, even advanced LLMs cannot follow complex instructions
  well, thus limiting the quality of generated data. In this work, we find that
  existing datasets inherently contain implicit complex constraints and propose a
  novel data generation technique, constraint back-translation. Specifically, we
  take the high-quality instruction-response pairs in existing datasets and only
  adopt advanced LLMs to add complex constraints already met by the responses to
  the instructions, which naturally reduces costs and data noise. In the
  experiments, we adopt Llama3-70B-Instruct to back-translate constraints and
  create a high-quality complex instruction-response dataset, named CRAB. We
  present that post-training on CRAB improves multiple backbone LLMs' complex
  instruction-following ability, evaluated on extensive instruction-following
  benchmarks. We further find that constraint back-translation also serves as a
  useful auxiliary training objective in post-training. Our code, data, and
  models will be released to facilitate future research.
paper_authors._id: 672442ec7b7aa5555e85b437
paper_authors.name: Yunjia Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-03T14:22:11.267Z
paper_authors._id: 6556cf2cee35f7d8bcf13bb3
paper_authors.avatarUrl: /avatars/0a945054cb4732c2ee7a5502c42628bf.svg
paper_authors.fullname: Qi Yunjia
paper_authors.type: user
paper_authors.user: Kikkk
paper_authors._id: 672442ec7b7aa5555e85b438
paper_authors.name: Hao Peng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-02T20:19:32.600Z
paper_authors._id: 625a5446f1063e7085d5178a
paper_authors.avatarUrl: /avatars/5e78186f13f74b14e01583e06ff6c4dc.svg
paper_authors.fullname: Hao Peng
paper_authors.type: user
paper_authors.user: Wesleythu
paper_authors._id: 672442ec7b7aa5555e85b439
paper_authors.name: Xiaozhi Wang
paper_authors._id: 672442ec7b7aa5555e85b43a
paper_authors.name: Bin Xu
paper_authors._id: 672442ec7b7aa5555e85b43b
paper_authors.name: Lei Hou
paper_authors._id: 672442ec7b7aa5555e85b43c
paper_authors.name: Juanzi Li
paper_title: Constraint Back-translation Improves Complex Instruction Following of
    Large Language Models
paper_summary: Large language models (LLMs) struggle to follow instructions with complex
  constraints in format, length, etc. Following the conventional
  instruction-tuning practice, previous works conduct post-training on complex
  instruction-response pairs generated by feeding complex instructions to
  advanced LLMs. However, even advanced LLMs cannot follow complex instructions
  well, thus limiting the quality of generated data. In this work, we find that
  existing datasets inherently contain implicit complex constraints and propose a
  novel data generation technique, constraint back-translation. Specifically, we
  take the high-quality instruction-response pairs in existing datasets and only
  adopt advanced LLMs to add complex constraints already met by the responses to
  the instructions, which naturally reduces costs and data noise. In the
  experiments, we adopt Llama3-70B-Instruct to back-translate constraints and
  create a high-quality complex instruction-response dataset, named CRAB. We
  present that post-training on CRAB improves multiple backbone LLMs' complex
  instruction-following ability, evaluated on extensive instruction-following
  benchmarks. We further find that constraint back-translation also serves as a
  useful auxiliary training objective in post-training. Our code, data, and
  models will be released to facilitate future research.
paper_ai_keywords: large language models (LLMs) instruction-tuning instruction-response pairs constraint back-translation high-quality instruction-response dataset CRAB backbone LLMs instruction-following benchmarks auxiliary training objective