Teaching with Lies: Curriculum DPO on Synthetic Negatives for
  Hallucination Detection
Aligning large language models (LLMs) to accurately detect hallucinations
remains a significant challenge due to the sophisticated nature of hallucinated
text. Recognizing that hallucinated samples typically exhibit higher deceptive
quality than traditional negative samples, we use these carefully engineered
hallucinations as negative examples in the DPO alignment procedure. Our method
incorporates a curriculum learning strategy, gradually transitioning the
training from easier samples, identified based on the greatest reduction in
probability scores from independent fact checking models, to progressively
harder ones. This structured difficulty scaling ensures stable and incremental
learning. Experimental evaluation demonstrates that our HaluCheck models,
trained with curriculum DPO approach and high quality negative samples,
significantly improves model performance across various metrics, achieving
improvements of upto 24% on difficult benchmarks like MedHallu and HaluEval.
Additionally, HaluCheck models demonstrate robustness in zero-shot settings,
significantly outperforming larger state-of-the-art models across various
benchmarks.
6833c8af029c4a53a60a5dfa
Shrey Pandit
extracted_pending
2025-05-26T01:49:36.568Z
648749094dea003c6dae810f
https://cdn-avatars.huggingface.co/v1/production/uploads/648749094dea003c6dae810f/gHUHSBt1zrt8wjO1YwTNu.jpeg
Shrey Pandit
user
SP2001
6833c8af029c4a53a60a5dfb
Ashwin Vinod
claimed_verified
2025-05-26T08:10:22.347Z
62fa7294363251ee40a41dba
/avatars/869c6de9a1cb2ded690ae56559916cae.svg
Ashwin V
user
ashwinnv
6833c8af029c4a53a60a5dfc
Liu Leqi
6833c8af029c4a53a60a5dfd
Ying Ding
Teaching with Lies: Curriculum DPO on Synthetic Negatives for
  Hallucination Detection
Aligning large language models (LLMs) to accurately detect hallucinations
remains a significant challenge due to the sophisticated nature of hallucinated
text. Recognizing that hallucinated samples typically exhibit higher deceptive
quality than traditional negative samples, we use these carefully engineered
hallucinations as negative examples in the DPO alignment procedure. Our method
incorporates a curriculum learning strategy, gradually transitioning the
training from easier samples, identified based on the greatest reduction in
probability scores from independent fact checking models, to progressively
harder ones. This structured difficulty scaling ensures stable and incremental
learning. Experimental evaluation demonstrates that our HaluCheck models,
trained with curriculum DPO approach and high quality negative samples,
significantly improves model performance across various metrics, achieving
improvements of upto 24% on difficult benchmarks like MedHallu and HaluEval.
Additionally, HaluCheck models demonstrate robustness in zero-shot settings,
significantly outperforming larger state-of-the-art models across various
benchmarks.
The use of carefully crafted hallucinations in a curriculum learning approach within the DPO alignment procedure significantly enhances LLMs' hallucination detection abilities.
LLMs hallucinations DPO alignment procedure curriculum learning probability scores fact checking models HaluCheck models MedHallu HaluEval zero-shot settings