KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for
  Coding
We introduce KodCode, a synthetic dataset that addresses the persistent
challenge of acquiring high-quality, verifiable training data across diverse
difficulties and domains for training Large Language Models for coding.
Existing code-focused resources typically fail to ensure either the breadth of
coverage (e.g., spanning simple coding tasks to advanced algorithmic problems)
or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises
question-solution-test triplets that are systematically validated via a
self-verification procedure. Our pipeline begins by synthesizing a broad range
of coding questions, then generates solutions and test cases with additional
attempts allocated to challenging problems. Finally, post-training data
synthesis is done by rewriting questions into diverse formats and generating
responses under a test-based reject sampling procedure from a reasoning model
(DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding
dataset. KodCode is suitable for supervised fine-tuning and the paired unit
tests also provide great potential for RL tuning. Fine-tuning experiments on
coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench)
demonstrate that KodCode-tuned models achieve state-of-the-art performance,
surpassing models like Qwen2.5-Coder-32B-Instruct and
DeepSeek-R1-Distill-Llama-70B.
67c907ea7568a12737ad4535
Zhangchen Xu
claimed_verified
2025-03-06T09:26:50.636Z
653df1323479e9ebbe3eb6cc
https://cdn-avatars.huggingface.co/v1/production/uploads/653df1323479e9ebbe3eb6cc/K_g-r1iMRNKj99LXPuYF3.jpeg
Zhangchen Xu
user
zhangchenxu
67c907ea7568a12737ad4536
Yang Liu
extracted_pending
2025-03-06T02:26:54.940Z
637c88b6d55081513c5690d8
/avatars/6766e23ebf46b46d6c8b48351c571907.svg
Yang Liu
user
nlpyang
67c907ea7568a12737ad4537
Yueqin Yin
claimed_verified
2025-03-06T09:26:48.614Z
605e8dfd5abeb13e714c4c18
/avatars/bc27a0ed17b2bd4311e89d3028fa327b.svg
yueqin yin
user
yyqoni
67c907ea7568a12737ad4538
Mingyuan Zhou
admin_assigned
2025-03-06T10:03:56.474Z
653b2524b77b5e255f2d29d2
/avatars/f69aea8de84c435295e7638bad5bd82e.svg
Mingyuan Zhou
user
mingyuanzhou
67c907ea7568a12737ad4539
Radha Poovendran
KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for
  Coding
We introduce KodCode, a synthetic dataset that addresses the persistent
challenge of acquiring high-quality, verifiable training data across diverse
difficulties and domains for training Large Language Models for coding.
Existing code-focused resources typically fail to ensure either the breadth of
coverage (e.g., spanning simple coding tasks to advanced algorithmic problems)
or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises
question-solution-test triplets that are systematically validated via a
self-verification procedure. Our pipeline begins by synthesizing a broad range
of coding questions, then generates solutions and test cases with additional
attempts allocated to challenging problems. Finally, post-training data
synthesis is done by rewriting questions into diverse formats and generating
responses under a test-based reject sampling procedure from a reasoning model
(DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding
dataset. KodCode is suitable for supervised fine-tuning and the paired unit
tests also provide great potential for RL tuning. Fine-tuning experiments on
coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench)
demonstrate that KodCode-tuned models achieve state-of-the-art performance,
surpassing models like Qwen2.5-Coder-32B-Instruct and
DeepSeek-R1-Distill-Llama-70B.
KodCode is a synthetic coding dataset that ensures broad coverage and correctness through systematic validation, enabling state-of-the-art performance in coding benchmarks.
Larger Language Models coding question-solution-test triplets self-verification procedure reasoning model DeepSeek R1 supervised fine-tuning RL tuning HumanEval MBPP BigCodeBench LiveCodeBench Qwen2.5-Coder-32B-Instruct DeepSeek-R1-Distill-Llama-70B