title: Teaching with Lies: Curriculum DPO on Synthetic Negatives for
    Hallucination Detection
summary: Aligning large language models (LLMs) to accurately detect hallucinations
  remains a significant challenge due to the sophisticated nature of hallucinated
  text. Recognizing that hallucinated samples typically exhibit higher deceptive
  quality than traditional negative samples, we use these carefully engineered
  hallucinations as negative examples in the DPO alignment procedure. Our method
  incorporates a curriculum learning strategy, gradually transitioning the
  training from easier samples, identified based on the greatest reduction in
  probability scores from independent fact checking models, to progressively
  harder ones. This structured difficulty scaling ensures stable and incremental
  learning. Experimental evaluation demonstrates that our HaluCheck models,
  trained with curriculum DPO approach and high quality negative samples,
  significantly improves model performance across various metrics, achieving
  improvements of upto 24% on difficult benchmarks like MedHallu and HaluEval.
  Additionally, HaluCheck models demonstrate robustness in zero-shot settings,
  significantly outperforming larger state-of-the-art models across various
  benchmarks.
paper_authors._id: 6833c8af029c4a53a60a5dfa
paper_authors.name: Shrey Pandit
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-26T01:49:36.568Z
paper_authors._id: 648749094dea003c6dae810f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648749094dea003c6dae810f/gHUHSBt1zrt8wjO1YwTNu.jpeg
paper_authors.fullname: Shrey Pandit
paper_authors.type: user
paper_authors.user: SP2001
paper_authors._id: 6833c8af029c4a53a60a5dfb
paper_authors.name: Ashwin Vinod
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:10:22.347Z
paper_authors._id: 62fa7294363251ee40a41dba
paper_authors.avatarUrl: /avatars/869c6de9a1cb2ded690ae56559916cae.svg
paper_authors.fullname: Ashwin V
paper_authors.type: user
paper_authors.user: ashwinnv
paper_authors._id: 6833c8af029c4a53a60a5dfc
paper_authors.name: Liu Leqi
paper_authors._id: 6833c8af029c4a53a60a5dfd
paper_authors.name: Ying Ding
paper_title: Teaching with Lies: Curriculum DPO on Synthetic Negatives for
    Hallucination Detection
paper_summary: Aligning large language models (LLMs) to accurately detect hallucinations
  remains a significant challenge due to the sophisticated nature of hallucinated
  text. Recognizing that hallucinated samples typically exhibit higher deceptive
  quality than traditional negative samples, we use these carefully engineered
  hallucinations as negative examples in the DPO alignment procedure. Our method
  incorporates a curriculum learning strategy, gradually transitioning the
  training from easier samples, identified based on the greatest reduction in
  probability scores from independent fact checking models, to progressively
  harder ones. This structured difficulty scaling ensures stable and incremental
  learning. Experimental evaluation demonstrates that our HaluCheck models,
  trained with curriculum DPO approach and high quality negative samples,
  significantly improves model performance across various metrics, achieving
  improvements of upto 24% on difficult benchmarks like MedHallu and HaluEval.
  Additionally, HaluCheck models demonstrate robustness in zero-shot settings,
  significantly outperforming larger state-of-the-art models across various
  benchmarks.
paper_ai_summary: The use of carefully crafted hallucinations in a curriculum learning approach within the DPO alignment procedure significantly enhances LLMs' hallucination detection abilities.
paper_ai_keywords: LLMs hallucinations DPO alignment procedure curriculum learning probability scores fact checking models HaluCheck models MedHallu HaluEval zero-shot settings