Can One Domain Help Others? A Data-Centric Study on Multi-Domain
  Reasoning via Reinforcement Learning
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a
powerful paradigm for enhancing the reasoning capabilities of LLMs. Existing
research has predominantly concentrated on isolated reasoning domains such as
mathematical problem-solving, coding tasks, or logical reasoning. However, real
world reasoning scenarios inherently demand an integrated application of
multiple cognitive skills. Despite this, the interplay among these reasoning
skills under reinforcement learning remains poorly understood. To bridge this
gap, we present a systematic investigation of multi-domain reasoning within the
RLVR framework, explicitly focusing on three primary domains: mathematical
reasoning, code generation, and logical puzzle solving. We conduct a
comprehensive study comprising four key components: (1) Leveraging the GRPO
algorithm and the Qwen-2.5-7B model family, our study thoroughly evaluates the
models' in-domain improvements and cross-domain generalization capabilities
when trained on single-domain datasets. (2) Additionally, we examine the
intricate interactions including mutual enhancements and conflicts that emerge
during combined cross-domain training. (3) To further understand the influence
of SFT on RL, we also analyze and compare performance differences between base
and instruct models under identical RL configurations. (4) Furthermore, we
delve into critical RL training details, systematically exploring the impacts
of curriculum learning strategies, variations in reward design, and
language-specific factors. Through extensive experiments, our results offer
significant insights into the dynamics governing domain interactions, revealing
key factors influencing both specialized and generalizable reasoning
performance. These findings provide valuable guidance for optimizing RL
methodologies to foster comprehensive, multi-domain reasoning capabilities in
LLMs.
6881a669df7c5aafaf37f0bc
Yu Li
admin_assigned
2025-07-24T12:02:03.468Z
671b852aa4fa4f8f5fb5404c
https://cdn-avatars.huggingface.co/v1/production/uploads/671b852aa4fa4f8f5fb5404c/TDLsgP8WgKW-qaA8Ys-iJ.jpeg
YU LI
user
yu0226
6881a669df7c5aafaf37f0bd
Zhuoshi Pan
6881a669df7c5aafaf37f0be
Honglin Lin
6881a669df7c5aafaf37f0bf
Mengyuan Sun
admin_assigned
2025-07-24T12:01:39.741Z
67ad790c2b28204981be8e24
https://cdn-avatars.huggingface.co/v1/production/uploads/67ad790c2b28204981be8e24/KstE5e5bUXXIvgPJqMO2B.jpeg
Mengyuan Sun
user
blue01223
6881a669df7c5aafaf37f0c0
Conghui He
6881a669df7c5aafaf37f0c1
Lijun Wu
Can One Domain Help Others? A Data-Centric Study on Multi-Domain
  Reasoning via Reinforcement Learning
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a
powerful paradigm for enhancing the reasoning capabilities of LLMs. Existing
research has predominantly concentrated on isolated reasoning domains such as
mathematical problem-solving, coding tasks, or logical reasoning. However, real
world reasoning scenarios inherently demand an integrated application of
multiple cognitive skills. Despite this, the interplay among these reasoning
skills under reinforcement learning remains poorly understood. To bridge this
gap, we present a systematic investigation of multi-domain reasoning within the
RLVR framework, explicitly focusing on three primary domains: mathematical
reasoning, code generation, and logical puzzle solving. We conduct a
comprehensive study comprising four key components: (1) Leveraging the GRPO
algorithm and the Qwen-2.5-7B model family, our study thoroughly evaluates the
models' in-domain improvements and cross-domain generalization capabilities
when trained on single-domain datasets. (2) Additionally, we examine the
intricate interactions including mutual enhancements and conflicts that emerge
during combined cross-domain training. (3) To further understand the influence
of SFT on RL, we also analyze and compare performance differences between base
and instruct models under identical RL configurations. (4) Furthermore, we
delve into critical RL training details, systematically exploring the impacts
of curriculum learning strategies, variations in reward design, and
language-specific factors. Through extensive experiments, our results offer
significant insights into the dynamics governing domain interactions, revealing
key factors influencing both specialized and generalizable reasoning
performance. These findings provide valuable guidance for optimizing RL
methodologies to foster comprehensive, multi-domain reasoning capabilities in
LLMs.
A study investigates multi-domain reasoning in Reinforcement Learning with Verifiable Rewards (RLVR) using the GRPO algorithm and Qwen-2.5-7B model family, exploring in-domain improvements, cross-domain generalization, and the impact of curriculum learning and reward design.
Reinforcement Learning with Verifiable Rewards RLVR GRPO algorithm Qwen-2.5-7B multi-domain reasoning in-domain improvements cross-domain generalization curriculum learning reward design