title: CLIPPER: Compression enables long-context synthetic data generation
summary: LLM developers are increasingly reliant on synthetic data, but generating
  high-quality data for complex long-context reasoning tasks remains challenging.
  We introduce CLIPPER, a compression-based approach for generating synthetic
  data tailored to narrative claim verification - a task that requires reasoning
  over a book to verify a given claim. Instead of generating claims directly from
  the raw text of the book, which results in artifact-riddled claims, CLIPPER
  first compresses the book into chapter outlines and book summaries and then
  uses these intermediate representations to generate complex claims and
  corresponding chain-of-thoughts. Compared to naive approaches, CLIPPER produces
  claims that are more valid, grounded, and complex. Using CLIPPER, we construct
  a dataset of 19K synthetic book claims paired with their source texts and
  chain-of-thought reasoning, and use it to fine-tune three open-weight models.
  Our best model achieves breakthrough results on narrative claim verification
  (from 28% to 76% accuracy on our test set) and sets a new state-of-the-art for
  sub-10B models on the NoCha leaderboard. Further analysis shows that our models
  generate more detailed and grounded chain-of-thought reasoning while also
  improving performance on other narrative understanding tasks (e.g.,
  NarrativeQA).
paper_authors._id: 67b7edf6a1d1394d1682c085
paper_authors.name: Chau Minh Pham
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-21T09:58:36.388Z
paper_authors._id: 65b976fdf69f4d0377aef3fe
paper_authors.avatarUrl: /avatars/1201194e2956c56b50098cc465a04c11.svg
paper_authors.fullname: Chau Minh Pham
paper_authors.type: user
paper_authors.user: chtmp223
paper_authors._id: 67b7edf6a1d1394d1682c086
paper_authors.name: Yapei Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T15:17:25.220Z
paper_authors._id: 5f1dcc06cb8f993fa01f4775
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668660782327-5f1dcc06cb8f993fa01f4775.png
paper_authors.fullname: Yapei Chang
paper_authors.type: user
paper_authors.user: yapeichang
paper_authors._id: 67b7edf6a1d1394d1682c087
paper_authors.name: Mohit Iyyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T15:17:30.925Z
paper_authors._id: 6669df71d1652853e4b66ee5
paper_authors.avatarUrl: /avatars/8a40bca9423ef76f39ae24d7a9e63478.svg
paper_authors.fullname: Mohit Iyyer
paper_authors.type: user
paper_authors.user: mohitiyyer
paper_title: CLIPPER: Compression enables long-context synthetic data generation
paper_summary: LLM developers are increasingly reliant on synthetic data, but generating
  high-quality data for complex long-context reasoning tasks remains challenging.
  We introduce CLIPPER, a compression-based approach for generating synthetic
  data tailored to narrative claim verification - a task that requires reasoning
  over a book to verify a given claim. Instead of generating claims directly from
  the raw text of the book, which results in artifact-riddled claims, CLIPPER
  first compresses the book into chapter outlines and book summaries and then
  uses these intermediate representations to generate complex claims and
  corresponding chain-of-thoughts. Compared to naive approaches, CLIPPER produces
  claims that are more valid, grounded, and complex. Using CLIPPER, we construct
  a dataset of 19K synthetic book claims paired with their source texts and
  chain-of-thought reasoning, and use it to fine-tune three open-weight models.
  Our best model achieves breakthrough results on narrative claim verification
  (from 28% to 76% accuracy on our test set) and sets a new state-of-the-art for
  sub-10B models on the NoCha leaderboard. Further analysis shows that our models
  generate more detailed and grounded chain-of-thought reasoning while also
  improving performance on other narrative understanding tasks (e.g.,
  NarrativeQA).
paper_ai_summary: CLIPPER compresses book content into outlines and summaries to generate high-quality synthetic claims for narrative claim verification, improving model accuracy and performance.
paper_ai_keywords: CLIPPER synthetic data narrative claim verification chapter outlines book summaries chain-of-thoughts fine-tuning NoCha leaderboard NarrativeQA