SAND-Math: Using LLMs to Generate Novel, Difficult and Useful
  Mathematics Questions and Answers
The demand for Large Language Models (LLMs) capable of sophisticated
mathematical reasoning is growing across industries. However, the development
of performant mathematical LLMs is critically bottlenecked by the scarcity of
difficult, novel training data. We introduce SAND-Math (Synthetic
Augmented Novel and Difficult Mathematics problems and solutions), a pipeline
that addresses this by first generating high-quality problems from scratch and
then systematically elevating their complexity via a new Difficulty
Hiking step. We demonstrate the effectiveness of our approach through two key
findings. First, augmenting a strong baseline with SAND-Math data significantly
boosts performance, outperforming the next-best synthetic dataset by
uparrow 17.85 absolute points on the AIME25 benchmark. Second, in a
dedicated ablation study, we show our Difficulty Hiking process is highly
effective: by increasing average problem difficulty from 5.02 to 5.98, this
step lifts AIME25 performance from 46.38\% to 49.23\%. The full generation
pipeline, final dataset, and a fine-tuned model form a practical and scalable
toolkit for building more capable and efficient mathematical reasoning LLMs.
SAND-Math dataset is released here:
https://huggingface.co/datasets/amd/SAND-MATH{https://huggingface.co/datasets/amd/SAND-MATH}
6888296daf872d625c10c5ab
Chaitanya Manem
6888296daf872d625c10c5ac
Pratik Prabhanjan Brahma
6888296daf872d625c10c5ad
Prakamya Mishra
claimed_verified
2025-07-30T09:05:20.847Z
65bb2a8400e143d672b170c9
/avatars/e1d745d12f0a4c8dae95d2f6983d5201.svg
Prakamya Mishra
user
Prakamya
6888296daf872d625c10c5ae
Zicheng Liu
6888296daf872d625c10c5af
Emad Barsoum
SAND-Math: Using LLMs to Generate Novel, Difficult and Useful
  Mathematics Questions and Answers
The demand for Large Language Models (LLMs) capable of sophisticated
mathematical reasoning is growing across industries. However, the development
of performant mathematical LLMs is critically bottlenecked by the scarcity of
difficult, novel training data. We introduce SAND-Math (Synthetic
Augmented Novel and Difficult Mathematics problems and solutions), a pipeline
that addresses this by first generating high-quality problems from scratch and
then systematically elevating their complexity via a new Difficulty
Hiking step. We demonstrate the effectiveness of our approach through two key
findings. First, augmenting a strong baseline with SAND-Math data significantly
boosts performance, outperforming the next-best synthetic dataset by
uparrow 17.85 absolute points on the AIME25 benchmark. Second, in a
dedicated ablation study, we show our Difficulty Hiking process is highly
effective: by increasing average problem difficulty from 5.02 to 5.98, this
step lifts AIME25 performance from 46.38\% to 49.23\%. The full generation
pipeline, final dataset, and a fine-tuned model form a practical and scalable
toolkit for building more capable and efficient mathematical reasoning LLMs.
SAND-Math dataset is released here:
https://huggingface.co/datasets/amd/SAND-MATH{https://huggingface.co/datasets/amd/SAND-MATH}
SAND-Math, a pipeline for generating and elevating the complexity of mathematical problems, significantly enhances the performance of Large Language Models on the AIME25 benchmark.
SAND-Math Synthetic Augmented Novel and Difficult Mathematics Difficulty Hiking Large Language Models AIME25 benchmark