title: CodecLM: Aligning Language Models with Tailored Synthetic Data
summary: Instruction tuning has emerged as the key in aligning large language models
  (LLMs) with specific task instructions, thereby mitigating the discrepancy
  between the next-token prediction objective and users' actual goals. To reduce
  the labor and time cost to collect or annotate data by humans, researchers
  start to explore the use of LLMs to generate instruction-aligned synthetic
  data. Recent works focus on generating diverse instructions and applying LLM to
  increase instruction complexity, often neglecting downstream use cases. It
  remains unclear how to tailor high-quality data to elicit better
  instruction-following abilities in different target instruction distributions
  and LLMs. To this end, we introduce CodecLM, a general framework for adaptively
  generating high-quality synthetic data for LLM alignment with different
  downstream instruction distributions and LLMs. Drawing on the Encode-Decode
  principles, we use LLMs as codecs to guide the data generation process. We
  first encode seed instructions into metadata, which are concise keywords
  generated on-the-fly to capture the target instruction distribution, and then
  decode metadata to create tailored instructions. We also introduce Self-Rubrics
  and Contrastive Filtering during decoding to tailor data-efficient samples.
  Extensive experiments on four open-domain instruction following benchmarks
  validate the effectiveness of CodecLM over the current state-of-the-arts.
paper_authors._id: 6615f4b861fb2574d52a37ed
paper_authors.name: Zifeng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-12T19:56:05.811Z
paper_authors._id: 64da7b3bbb090cef5559fa31
paper_authors.avatarUrl: /avatars/4eb059260f06837192c98927254cd89e.svg
paper_authors.fullname: Zifeng Wang
paper_authors.type: user
paper_authors.user: zifengw
paper_authors._id: 6615f4b861fb2574d52a37ee
paper_authors.name: Chun-Liang Li
paper_authors._id: 6615f4b861fb2574d52a37ef
paper_authors.name: Vincent Perot
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-11T05:52:30.935Z
paper_authors._id: 64e3f501bbee0c24b95e20c2
paper_authors.avatarUrl: /avatars/bae7b5297e779d91da21ff50389dd90a.svg
paper_authors.fullname: Vincent
paper_authors.type: user
paper_authors.user: vperot
paper_authors._id: 6615f4b861fb2574d52a37f0
paper_authors.name: Long T. Le
paper_authors._id: 6615f4b861fb2574d52a37f1
paper_authors.name: Jin Miao
paper_authors._id: 6615f4b861fb2574d52a37f2
paper_authors.name: Zizhao Zhang
paper_authors._id: 6615f4b861fb2574d52a37f3
paper_authors.name: Chen-Yu Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-24T07:29:12.475Z
paper_authors._id: 64d1fe784dfd5df7076bfd1a
paper_authors.avatarUrl: /avatars/e2bb108d1f6b1383f0e3f263c4747b3d.svg
paper_authors.fullname: Chen-Yu Lee
paper_authors.type: user
paper_authors.user: chenyulee
paper_authors._id: 6615f4b861fb2574d52a37f4
paper_authors.name: Tomas Pfister
paper_title: CodecLM: Aligning Language Models with Tailored Synthetic Data
paper_summary: Instruction tuning has emerged as the key in aligning large language models
  (LLMs) with specific task instructions, thereby mitigating the discrepancy
  between the next-token prediction objective and users' actual goals. To reduce
  the labor and time cost to collect or annotate data by humans, researchers
  start to explore the use of LLMs to generate instruction-aligned synthetic
  data. Recent works focus on generating diverse instructions and applying LLM to
  increase instruction complexity, often neglecting downstream use cases. It
  remains unclear how to tailor high-quality data to elicit better
  instruction-following abilities in different target instruction distributions
  and LLMs. To this end, we introduce CodecLM, a general framework for adaptively
  generating high-quality synthetic data for LLM alignment with different
  downstream instruction distributions and LLMs. Drawing on the Encode-Decode
  principles, we use LLMs as codecs to guide the data generation process. We
  first encode seed instructions into metadata, which are concise keywords
  generated on-the-fly to capture the target instruction distribution, and then
  decode metadata to create tailored instructions. We also introduce Self-Rubrics
  and Contrastive Filtering during decoding to tailor data-efficient samples.
  Extensive experiments on four open-domain instruction following benchmarks
  validate the effectiveness of CodecLM over the current state-of-the-arts.
paper_ai_summary: CodecLM is a framework that utilizes LLMs to generate high-quality, aligned synthetic data for instruction-following tasks, improving adaptability and performance across various LLMs and instruction distributions.
paper_ai_keywords: instruction tuning large language models (LLMs) synthetic data Encode-Decode principles codecs metadata self-rubrics contrastive filtering instruction-following benchmarks