title: Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and
    Refinement
summary: The quality of Supervised Fine-Tuning (SFT) data plays a critical role in
  enhancing the conversational capabilities of Large Language Models (LLMs).
  However, as LLMs become more advanced, the availability of high-quality
  human-annotated SFT data has become a significant bottleneck, necessitating a
  greater reliance on synthetic training data. In this work, we introduce Condor,
  a novel two-stage synthetic data generation framework that incorporates World
  Knowledge Tree and Self-Reflection Refinement to produce high-quality SFT data
  at scale. Our experimental results demonstrate that a base model fine-tuned on
  only 20K Condor-generated samples achieves superior performance compared to
  counterparts. The additional refinement stage in Condor further enables
  iterative self-improvement for LLMs at various scales (up to 72B), validating
  the effectiveness of our approach. Furthermore, our investigation into the
  scaling for synthetic data in post-training reveals substantial unexplored
  potential for performance improvements, opening promising avenues for future
  research.
paper_authors._id: 67906c674932687e24e0cc08
paper_authors.name: Maosong Cao
paper_authors._id: 67906c674932687e24e0cc09
paper_authors.name: Taolin Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T16:00:36.088Z
paper_authors._id: 62a2b0f3575099d76bdc7259
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654829516458-62a2b0f3575099d76bdc7259.png
paper_authors.fullname: Taolin Zhang
paper_authors.type: user
paper_authors.user: ruxian
paper_authors._id: 67906c674932687e24e0cc0a
paper_authors.name: Mo Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T16:01:30.729Z
paper_authors._id: 674fe4239cad8e296c9f2d1e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iZ22kVJClIzEK35oxzsp3.png
paper_authors.fullname: Mo Li
paper_authors.type: user
paper_authors.user: MoLi-DHU
paper_authors._id: 67906c674932687e24e0cc0b
paper_authors.name: Chuyu Zhang
paper_authors._id: 67906c674932687e24e0cc0c
paper_authors.name: Yunxin Liu
paper_authors._id: 67906c674932687e24e0cc0d
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T16:01:03.878Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 67906c674932687e24e0cc0e
paper_authors.name: Songyang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-22T10:08:23.652Z
paper_authors._id: 630716d11801ecc7d2595021
paper_authors.avatarUrl: /avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
paper_authors.fullname: Songyang Zhang
paper_authors.type: user
paper_authors.user: zsytony
paper_authors._id: 67906c674932687e24e0cc0f
paper_authors.name: Kai Chen
paper_title: Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and
    Refinement
paper_summary: The quality of Supervised Fine-Tuning (SFT) data plays a critical role in
  enhancing the conversational capabilities of Large Language Models (LLMs).
  However, as LLMs become more advanced, the availability of high-quality
  human-annotated SFT data has become a significant bottleneck, necessitating a
  greater reliance on synthetic training data. In this work, we introduce Condor,
  a novel two-stage synthetic data generation framework that incorporates World
  Knowledge Tree and Self-Reflection Refinement to produce high-quality SFT data
  at scale. Our experimental results demonstrate that a base model fine-tuned on
  only 20K Condor-generated samples achieves superior performance compared to
  counterparts. The additional refinement stage in Condor further enables
  iterative self-improvement for LLMs at various scales (up to 72B), validating
  the effectiveness of our approach. Furthermore, our investigation into the
  scaling for synthetic data in post-training reveals substantial unexplored
  potential for performance improvements, opening promising avenues for future
  research.
paper_ai_summary: Condor, a two-stage synthetic data generation framework, enhances the performance of large language models through World Knowledge Tree and Self-Reflection Refinement, generating high-quality supervised fine-tuning data.
paper_ai_keywords: Supervised Fine-Tuning Large Language Models Condor World Knowledge Tree Self-Reflection Refinement