title: KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for
    Coding
summary: We introduce KodCode, a synthetic dataset that addresses the persistent
  challenge of acquiring high-quality, verifiable training data across diverse
  difficulties and domains for training Large Language Models for coding.
  Existing code-focused resources typically fail to ensure either the breadth of
  coverage (e.g., spanning simple coding tasks to advanced algorithmic problems)
  or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises
  question-solution-test triplets that are systematically validated via a
  self-verification procedure. Our pipeline begins by synthesizing a broad range
  of coding questions, then generates solutions and test cases with additional
  attempts allocated to challenging problems. Finally, post-training data
  synthesis is done by rewriting questions into diverse formats and generating
  responses under a test-based reject sampling procedure from a reasoning model
  (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding
  dataset. KodCode is suitable for supervised fine-tuning and the paired unit
  tests also provide great potential for RL tuning. Fine-tuning experiments on
  coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench)
  demonstrate that KodCode-tuned models achieve state-of-the-art performance,
  surpassing models like Qwen2.5-Coder-32B-Instruct and
  DeepSeek-R1-Distill-Llama-70B.
paper_authors._id: 67c907ea7568a12737ad4535
paper_authors.name: Zhangchen Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-06T09:26:50.636Z
paper_authors._id: 653df1323479e9ebbe3eb6cc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/653df1323479e9ebbe3eb6cc/K_g-r1iMRNKj99LXPuYF3.jpeg
paper_authors.fullname: Zhangchen Xu
paper_authors.type: user
paper_authors.user: zhangchenxu
paper_authors._id: 67c907ea7568a12737ad4536
paper_authors.name: Yang Liu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-06T02:26:54.940Z
paper_authors._id: 637c88b6d55081513c5690d8
paper_authors.avatarUrl: /avatars/6766e23ebf46b46d6c8b48351c571907.svg
paper_authors.fullname: Yang Liu
paper_authors.type: user
paper_authors.user: nlpyang
paper_authors._id: 67c907ea7568a12737ad4537
paper_authors.name: Yueqin Yin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-06T09:26:48.614Z
paper_authors._id: 605e8dfd5abeb13e714c4c18
paper_authors.avatarUrl: /avatars/bc27a0ed17b2bd4311e89d3028fa327b.svg
paper_authors.fullname: yueqin yin
paper_authors.type: user
paper_authors.user: yyqoni
paper_authors._id: 67c907ea7568a12737ad4538
paper_authors.name: Mingyuan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-06T10:03:56.474Z
paper_authors._id: 653b2524b77b5e255f2d29d2
paper_authors.avatarUrl: /avatars/f69aea8de84c435295e7638bad5bd82e.svg
paper_authors.fullname: Mingyuan Zhou
paper_authors.type: user
paper_authors.user: mingyuanzhou
paper_authors._id: 67c907ea7568a12737ad4539
paper_authors.name: Radha Poovendran
paper_title: KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for
    Coding
paper_summary: We introduce KodCode, a synthetic dataset that addresses the persistent
  challenge of acquiring high-quality, verifiable training data across diverse
  difficulties and domains for training Large Language Models for coding.
  Existing code-focused resources typically fail to ensure either the breadth of
  coverage (e.g., spanning simple coding tasks to advanced algorithmic problems)
  or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises
  question-solution-test triplets that are systematically validated via a
  self-verification procedure. Our pipeline begins by synthesizing a broad range
  of coding questions, then generates solutions and test cases with additional
  attempts allocated to challenging problems. Finally, post-training data
  synthesis is done by rewriting questions into diverse formats and generating
  responses under a test-based reject sampling procedure from a reasoning model
  (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding
  dataset. KodCode is suitable for supervised fine-tuning and the paired unit
  tests also provide great potential for RL tuning. Fine-tuning experiments on
  coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench)
  demonstrate that KodCode-tuned models achieve state-of-the-art performance,
  surpassing models like Qwen2.5-Coder-32B-Instruct and
  DeepSeek-R1-Distill-Llama-70B.
paper_ai_keywords: Large Language Models self-verification procedure test-based reject sampling DeepSeek R1 reasoning model