title: CritiQ: Mining Data Quality Criteria from Human Preferences
summary: Language model heavily depends on high-quality data for optimal performance.
  Existing approaches rely on manually designed heuristics, the perplexity of
  existing models, training classifiers, or careful prompt engineering, which
  require significant expert experience and human annotation effort while
  introduce biases. We introduce CritiQ, a novel data selection method that
  automatically mines criteria from human preferences for data quality with only
  sim30 human-annotated pairs and performs efficient data selection. The main
  component, CritiQ Flow, employs a manager agent to evolve quality criteria and
  worker agents to make pairwise judgments. We build a knowledge base that
  extracts quality criteria from previous work to boost CritiQ Flow. Compared to
  perplexity- and classifier- based methods, verbal criteria are more
  interpretable and possess reusable value. After deriving the criteria, we train
  the CritiQ Scorer to give quality scores and perform efficient data selection.
  We demonstrate the effectiveness of our method in the code, math, and logic
  domains, achieving high accuracy on human-annotated test sets. To validate the
  quality of the selected data, we continually train Llama 3.1 models and observe
  improved performance on downstream tasks compared to uniform sampling. Ablation
  studies validate the benefits of the knowledge base and the reflection process.
  We analyze how criteria evolve and the effectiveness of majority voting.
paper_authors._id: 67bffaca3f838c1e33e074e7
paper_authors.name: Honglin Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:13:52.094Z
paper_authors._id: 638ef0b0c67af472d31674a6
paper_authors.avatarUrl: /avatars/02df97d15a0f46b47f9162221733b121.svg
paper_authors.fullname: Honglin Guo
paper_authors.type: user
paper_authors.user: KYLN24
paper_authors._id: 67bffaca3f838c1e33e074e8
paper_authors.name: Kai Lv
paper_authors._id: 67bffaca3f838c1e33e074e9
paper_authors.name: Qipeng Guo
paper_authors._id: 67bffaca3f838c1e33e074ea
paper_authors.name: Tianyi Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:36:33.906Z
paper_authors._id: 62c14609ac1b639c2d87192c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656833489364-noauth.png
paper_authors.fullname: liangtianyi
paper_authors.type: user
paper_authors.user: tianyilt
paper_authors._id: 67bffaca3f838c1e33e074eb
paper_authors.name: Zhiheng Xi
paper_authors._id: 67bffaca3f838c1e33e074ec
paper_authors.name: Demin Song
paper_authors._id: 67bffaca3f838c1e33e074ed
paper_authors.name: Qiuyinzhe Zhang
paper_authors._id: 67bffaca3f838c1e33e074ee
paper_authors.name: Yu Sun
paper_authors._id: 67bffaca3f838c1e33e074ef
paper_authors.name: Kai Chen
paper_authors._id: 67bffaca3f838c1e33e074f0
paper_authors.name: Xipeng Qiu
paper_authors._id: 67bffaca3f838c1e33e074f1
paper_authors.name: Tao Gui
paper_title: CritiQ: Mining Data Quality Criteria from Human Preferences
paper_summary: Language model heavily depends on high-quality data for optimal performance.
  Existing approaches rely on manually designed heuristics, the perplexity of
  existing models, training classifiers, or careful prompt engineering, which
  require significant expert experience and human annotation effort while
  introduce biases. We introduce CritiQ, a novel data selection method that
  automatically mines criteria from human preferences for data quality with only
  sim30 human-annotated pairs and performs efficient data selection. The main
  component, CritiQ Flow, employs a manager agent to evolve quality criteria and
  worker agents to make pairwise judgments. We build a knowledge base that
  extracts quality criteria from previous work to boost CritiQ Flow. Compared to
  perplexity- and classifier- based methods, verbal criteria are more
  interpretable and possess reusable value. After deriving the criteria, we train
  the CritiQ Scorer to give quality scores and perform efficient data selection.
  We demonstrate the effectiveness of our method in the code, math, and logic
  domains, achieving high accuracy on human-annotated test sets. To validate the
  quality of the selected data, we continually train Llama 3.1 models and observe
  improved performance on downstream tasks compared to uniform sampling. Ablation
  studies validate the benefits of the knowledge base and the reflection process.
  We analyze how criteria evolve and the effectiveness of majority voting.
paper_ai_summary: CritiQ automates data selection using human preferences, improving language model performance through efficient and interpretable criteria.
paper_ai_keywords: CritiQ CritiQ Flow manager agent worker agents quality criteria knowledge base CritiQ Scorer data selection Llama 3.1 models downstream tasks majority voting