title: SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis
summary: Vision-language models (VLMs) trained via reinforcement learning with
  verifiable reward (RLVR) have shown notable progress in scaling test-time
  compute effectively. In this work, we investigate how synthesized RL data can
  further improve RLVR. To this end, we propose SynthRL-a scalable and
  guaranteed pipeline for automatic data scaling in reasoning-oriented RL
  training. SynthRL comprises three key stages: (1) selecting seed questions with
  appropriate distribution, (2) augmenting them into more challenging variants
  while preserving the original answers, and (3) a guaranteed verification stage
  that ensures near-perfect correctness and difficulty enhancement. Our empirical
  experiments demonstrate SynthRL's scalability and effectiveness. When applied
  to the MMK12 dataset, SynthRL synthesizes over 3.3K additional verifiable,
  challenging questions from approximately 8K seed samples. Models trained with
  our synthesized data achieve consistent gains across five out-of-domain visual
  math reasoning benchmarks, with a significant improvement over baseline models
  trained on seed data alone. Notably, detailed analysis reveals that the gains
  are more pronounced on the most challenging evaluation samples, highlighting
  SynthRL's effectiveness in eliciting deeper and more complex reasoning
  patterns.
paper_authors._id: 683fa36f7ed99d0040761114
paper_authors.name: Zijian Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:57:09.765Z
paper_authors._id: 626d268d5f7327906f05cad1
paper_authors.avatarUrl: /avatars/18bda74612a3ee63a17f991bcc695106.svg
paper_authors.fullname: Zijian Wu
paper_authors.type: user
paper_authors.user: Jakumetsu
paper_authors._id: 683fa36f7ed99d0040761115
paper_authors.name: Jinjie Ni
paper_authors._id: 683fa36f7ed99d0040761116
paper_authors.name: Xiangyan Liu
paper_authors._id: 683fa36f7ed99d0040761117
paper_authors.name: Zichen Liu
paper_authors._id: 683fa36f7ed99d0040761118
paper_authors.name: Hang Yan
paper_authors._id: 683fa36f7ed99d0040761119
paper_authors.name: Michael Qizhe Shieh
paper_title: SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis
paper_summary: Vision-language models (VLMs) trained via reinforcement learning with
  verifiable reward (RLVR) have shown notable progress in scaling test-time
  compute effectively. In this work, we investigate how synthesized RL data can
  further improve RLVR. To this end, we propose SynthRL-a scalable and
  guaranteed pipeline for automatic data scaling in reasoning-oriented RL
  training. SynthRL comprises three key stages: (1) selecting seed questions with
  appropriate distribution, (2) augmenting them into more challenging variants
  while preserving the original answers, and (3) a guaranteed verification stage
  that ensures near-perfect correctness and difficulty enhancement. Our empirical
  experiments demonstrate SynthRL's scalability and effectiveness. When applied
  to the MMK12 dataset, SynthRL synthesizes over 3.3K additional verifiable,
  challenging questions from approximately 8K seed samples. Models trained with
  our synthesized data achieve consistent gains across five out-of-domain visual
  math reasoning benchmarks, with a significant improvement over baseline models
  trained on seed data alone. Notably, detailed analysis reveals that the gains
  are more pronounced on the most challenging evaluation samples, highlighting
  SynthRL's effectiveness in eliciting deeper and more complex reasoning
  patterns.
paper_ai_summary: SynthRL, a scalable pipeline for data synthesis in reinforcement learning with verifiable rewards, enhances visual math reasoning VLMs by generating challenging, verifiable questions.
paper_ai_keywords: reinforcement learning verifiable reward RLVR SynthRL seed questions data augmentation verification stage MMK12 dataset visual math reasoning benchmarks