MegaScience: Pushing the Frontiers of Post-Training Datasets for Science
  Reasoning
Scientific reasoning is critical for developing AI scientists and supporting
human researchers in advancing the frontiers of natural science discovery.
However, the open-source community has primarily focused on mathematics and
coding while neglecting the scientific domain, largely due to the absence of
open, large-scale, high-quality, verifiable scientific reasoning datasets. To
bridge this gap, we first present TextbookReasoning, an open dataset featuring
truthful reference answers extracted from 12k university-level scientific
textbooks, comprising 650k reasoning questions spanning 7 scientific
disciplines. We further introduce MegaScience, a large-scale mixture of
high-quality open-source datasets totaling 1.25 million instances, developed
through systematic ablation studies that evaluate various data selection
methodologies to identify the optimal subset for each publicly available
scientific dataset. Meanwhile, we build a comprehensive evaluation system
covering diverse subjects and question types across 15 benchmarks,
incorporating comprehensive answer extraction strategies to ensure accurate
evaluation metrics. Our experiments demonstrate that our datasets achieve
superior performance and training efficiency with more concise response lengths
compared to existing open-source scientific datasets. Furthermore, we train
Llama3.1, Qwen2.5, and Qwen3 series base models on MegaScience, which
significantly outperform the corresponding official instruct models in average
performance. In addition, MegaScience exhibits greater effectiveness for larger
and stronger models, suggesting a scaling benefit for scientific tuning. We
release our data curation pipeline, evaluation system, datasets, and seven
trained models to the community to advance scientific reasoning research.
688037ee00f4b5a05f2fbda9
Run-Ze Fan
claimed_verified
2025-07-23T08:36:54.362Z
616bfc2b40e2f69baa1c7add
https://cdn-avatars.huggingface.co/v1/production/uploads/616bfc2b40e2f69baa1c7add/Os7_qgMei-2lRVelrOG7B.jpeg
Run-Ze Fan
user
Vfrz
688037ee00f4b5a05f2fbdaa
Zengzhi Wang
claimed_verified
2025-07-23T08:36:51.124Z
62cbeb2d72dfd24b86bdf977
https://cdn-avatars.huggingface.co/v1/production/uploads/62cbeb2d72dfd24b86bdf977/UcGYYSBNrCvPM5K9v-sro.png
Zengzhi Wang
user
SinclairWang
688037ee00f4b5a05f2fbdab
Pengfei Liu
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science
  Reasoning
Scientific reasoning is critical for developing AI scientists and supporting
human researchers in advancing the frontiers of natural science discovery.
However, the open-source community has primarily focused on mathematics and
coding while neglecting the scientific domain, largely due to the absence of
open, large-scale, high-quality, verifiable scientific reasoning datasets. To
bridge this gap, we first present TextbookReasoning, an open dataset featuring
truthful reference answers extracted from 12k university-level scientific
textbooks, comprising 650k reasoning questions spanning 7 scientific
disciplines. We further introduce MegaScience, a large-scale mixture of
high-quality open-source datasets totaling 1.25 million instances, developed
through systematic ablation studies that evaluate various data selection
methodologies to identify the optimal subset for each publicly available
scientific dataset. Meanwhile, we build a comprehensive evaluation system
covering diverse subjects and question types across 15 benchmarks,
incorporating comprehensive answer extraction strategies to ensure accurate
evaluation metrics. Our experiments demonstrate that our datasets achieve
superior performance and training efficiency with more concise response lengths
compared to existing open-source scientific datasets. Furthermore, we train
Llama3.1, Qwen2.5, and Qwen3 series base models on MegaScience, which
significantly outperform the corresponding official instruct models in average
performance. In addition, MegaScience exhibits greater effectiveness for larger
and stronger models, suggesting a scaling benefit for scientific tuning. We
release our data curation pipeline, evaluation system, datasets, and seven
trained models to the community to advance scientific reasoning research.
TextbookReasoning and MegaScience datasets, along with a comprehensive evaluation system, enhance scientific reasoning in AI by providing high-quality, verifiable data and outperforming existing models.
TextbookReasoning MegaScience scientific reasoning datasets university-level scientific textbooks reasoning questions data selection methodologies evaluation system benchmarks answer extraction strategies Llama3.1 Qwen2.5 Qwen3 series scientific tuning