title: CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom
summary: Distilling advanced Large Language Models' instruction-following capabilities
  into smaller models using a selected subset has become a mainstream approach in
  model training. While existing synthetic instruction data selection strategies
  rely mainly on single-dimensional signals (i.e., reward scores, model
  perplexity), they fail to capture the complexity of instruction-following
  across diverse fields. Therefore, we investigate more diverse signals to
  capture comprehensive instruction-response pair characteristics and propose
  three foundational metrics that leverage Multi-LLM wisdom, informed by (1)
  diverse LLM responses and (2) reward model assessment. Building upon base
  metrics, we propose CrowdSelect, an integrated metric incorporating a
  clustering-based approach to maintain response diversity. Our comprehensive
  experiments demonstrate that our foundation metrics consistently improve
  performance across 4 base models on MT-bench and Arena-Hard. CrowdSelect,
  efficiently incorporating all metrics, achieves state-of-the-art performance in
  both Full and LoRA fine-tuning, showing improvements of 4.81% on Arena-Hard and
  11.1% on MT-bench with Llama-3.2-3b-instruct. We hope our findings will bring
  valuable insights for future research in this direction. Code are available at
  https://github.com/listentm/crowdselect.
paper_authors._id: 67c94c32dd505e6a4db201a2
paper_authors.name: Yisen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T10:04:09.687Z
paper_authors._id: 65d9a453f20e4fc19480afba
paper_authors.avatarUrl: /avatars/27bfa034a13a5e7cb5fc3b647515a201.svg
paper_authors.fullname: yisen li
paper_authors.type: user
paper_authors.user: yisenL
paper_authors._id: 67c94c32dd505e6a4db201a3
paper_authors.name: Lingfeng Yang
paper_authors._id: 67c94c32dd505e6a4db201a4
paper_authors.name: Wenxuan Shen
paper_authors._id: 67c94c32dd505e6a4db201a5
paper_authors.name: Pan Zhou
paper_authors._id: 67c94c32dd505e6a4db201a6
paper_authors.name: Yao Wan
paper_authors._id: 67c94c32dd505e6a4db201a7
paper_authors.name: Weiwei Lin
paper_authors._id: 67c94c32dd505e6a4db201a8
paper_authors.name: Dongping Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T10:09:00.496Z
paper_authors._id: 643be8879f5d314db2d9ed23
paper_authors.avatarUrl: /avatars/64e9bb2c4e10fbe03e2b81afedf40865.svg
paper_authors.fullname: Chen Dongping
paper_authors.type: user
paper_authors.user: shuaishuaicdp
paper_title: CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom
paper_summary: Distilling advanced Large Language Models' instruction-following capabilities
  into smaller models using a selected subset has become a mainstream approach in
  model training. While existing synthetic instruction data selection strategies
  rely mainly on single-dimensional signals (i.e., reward scores, model
  perplexity), they fail to capture the complexity of instruction-following
  across diverse fields. Therefore, we investigate more diverse signals to
  capture comprehensive instruction-response pair characteristics and propose
  three foundational metrics that leverage Multi-LLM wisdom, informed by (1)
  diverse LLM responses and (2) reward model assessment. Building upon base
  metrics, we propose CrowdSelect, an integrated metric incorporating a
  clustering-based approach to maintain response diversity. Our comprehensive
  experiments demonstrate that our foundation metrics consistently improve
  performance across 4 base models on MT-bench and Arena-Hard. CrowdSelect,
  efficiently incorporating all metrics, achieves state-of-the-art performance in
  both Full and LoRA fine-tuning, showing improvements of 4.81% on Arena-Hard and
  11.1% on MT-bench with Llama-3.2-3b-instruct. We hope our findings will bring
  valuable insights for future research in this direction. Code are available at
  https://github.com/listentm/crowdselect.
paper_ai_summary: The paper explores using a multi-dimensional approach, informed by multiple LLMs and a reward model, to select high-quality instruction data, improving smaller models' performance in instruction-following tasks.
paper_ai_keywords: Large Language Models instruction-following synthetic instruction data reward scores model perplexity multi-dimensional signals Multi-LLM wisdom LLM responses reward model assessment CrowdSelect clustering-based approach response diversity MT-bench Arena-Hard Full fine-tuning LoRA fine-tuning