NEMOTRON-CROSSTHINK: Scaling Self-Learning beyond Math Reasoning
Large Language Models (LLMs) have shown strong reasoning capabilities,
particularly when enhanced through Reinforcement Learning (RL). While prior
work has successfully applied RL to mathematical reasoning -- where rules and
correctness are well-defined -- generalizing these methods to broader reasoning
domains remains challenging due to limited data, the lack of verifiable reward
structures, and diverse task requirements. In this work, we propose
NEMOTRON-CROSSTHINK, a framework that systematically incorporates multi-domain
corpora, including both synthetic and real-world question-answer pairs, into RL
training to improve generalization across diverse reasoning tasks.
NEMOTRON-CROSSTHINK addresses key challenges by (1) incorporating data from
varied sources spanning STEM, humanities, social sciences, etc.; (2) applying
structured templates (e.g., multiple-choice and open-ended) to control
answer-space complexity; (3) filtering for verifiable answers; and (4)
optimizing data blending strategies that utilizes data from multiple sources
effectively. Our approach enables scalable and verifiable reward modeling
beyond mathematics and demonstrates improved accuracies on both math (MATH-500:
+30.1%, AMC23:+27.5%) and non-math reasoning benchmarks (MMLU-PRO: +12.8%,
GPQA-DIAMOND: +11.3%, AGIEVAL: +15.1%, SUPERGPQA: +3.8%). Moreover,
NEMOTRON-CROSSTHINK exhibits significantly improved response efficiency --
using 28% fewer tokens for correct answers -- highlighting more focused and
effective reasoning. Through NEMOTRON-CROSSTHINK, we demonstrate that
integrating multi-domain, multi-format data in RL leads to more accurate,
efficient, and generalizable LLMs.
6806f6ff67a715240a5ab9f8
Syeda Nahida Akter
admin_assigned
2025-04-22T11:06:38.568Z
6338dd1776421c0543150467
/avatars/4539dcec644e40be33f4a0d419fa66cb.svg
Syeda Nahida Akter
user
SieraL
6806f6ff67a715240a5ab9f9
Shrimai Prabhumoye
extracted_pending
2025-04-22T01:55:12.561Z
66980b9c9baa4382e1678809
/avatars/1a516bb7aa7871834c19de708cdd853a.svg
Shrimai Prabhumoye
user
shrimai19
6806f6ff67a715240a5ab9fa
Matvei Novikov
6806f6ff67a715240a5ab9fb
Seungju Han
6806f6ff67a715240a5ab9fc
Ying Lin
6806f6ff67a715240a5ab9fd
Evelina Bakhturi
admin_assigned
2025-04-22T11:06:52.587Z
64ecf888961565b46783bf08
/avatars/0c4ac90b7fa8a6091a67b0216dada7db.svg
Evelina Bakhturina
user
ekmb
6806f6ff67a715240a5ab9fe
Eric Nyberg
admin_assigned
2025-04-22T11:06:58.156Z
651b3cff66fe9bc55b2828d0
/avatars/1552a97628364537ae3db9fdb30b35c9.svg
Eric Nyberg
user
ericnyberg
6806f6ff67a715240a5ab9ff
Yejin Choi
admin_assigned
2025-04-22T11:07:04.801Z
64d42729f63b01b7f676b176
/avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
Yejin Choi
user
yejinchoinka
6806f6ff67a715240a5aba00
Mostofa Patwary
admin_assigned
2025-04-22T11:07:11.089Z
630544b09d2531fabd156fd3
/avatars/7b5374244a887577834fb4524ff76d01.svg
Mostofa Patwary
user
mpatwary
6806f6ff67a715240a5aba01
Mohammad Shoeybi
admin_assigned
2025-04-22T11:07:16.923Z
6641544c695975af2cbd0da6
/avatars/0ad3c18dcba585259b064fe9b00a07ce.svg
Mohammad Shoeybi
user
shoeybi
6806f6ff67a715240a5aba02
Bryan Catanzaro
admin_assigned
2025-04-22T11:07:23.267Z
6311021788942700629e6247
/avatars/e7adc1632b76e80e7e4a590033d1c20a.svg
Bryan Catanzaro
user
ctnzr
NEMOTRON-CROSSTHINK: Scaling Self-Learning beyond Math Reasoning
Large Language Models (LLMs) have shown strong reasoning capabilities,
particularly when enhanced through Reinforcement Learning (RL). While prior
work has successfully applied RL to mathematical reasoning -- where rules and
correctness are well-defined -- generalizing these methods to broader reasoning
domains remains challenging due to limited data, the lack of verifiable reward
structures, and diverse task requirements. In this work, we propose
NEMOTRON-CROSSTHINK, a framework that systematically incorporates multi-domain
corpora, including both synthetic and real-world question-answer pairs, into RL
training to improve generalization across diverse reasoning tasks.
NEMOTRON-CROSSTHINK addresses key challenges by (1) incorporating data from
varied sources spanning STEM, humanities, social sciences, etc.; (2) applying
structured templates (e.g., multiple-choice and open-ended) to control
answer-space complexity; (3) filtering for verifiable answers; and (4)
optimizing data blending strategies that utilizes data from multiple sources
effectively. Our approach enables scalable and verifiable reward modeling
beyond mathematics and demonstrates improved accuracies on both math (MATH-500:
+30.1%, AMC23:+27.5%) and non-math reasoning benchmarks (MMLU-PRO: +12.8%,
GPQA-DIAMOND: +11.3%, AGIEVAL: +15.1%, SUPERGPQA: +3.8%). Moreover,
NEMOTRON-CROSSTHINK exhibits significantly improved response efficiency --
using 28% fewer tokens for correct answers -- highlighting more focused and
effective reasoning. Through NEMOTRON-CROSSTHINK, we demonstrate that
integrating multi-domain, multi-format data in RL leads to more accurate,
efficient, and generalizable LLMs.
NEMOTRON-CROSSTHINK is a framework that incorporates diverse multi-domain data into RL training to enhance reasoning capabilities and efficiency of LLMs across various tasks and domains.
Reinforcement Learning LLMs NEMOTRON-CROSSTHINK multi-domain corpora structured templates reward modeling reasoning benchmarks MATH-500 AMC23 MMLU-PRO GPQA-DIAMOND AGIEVAL SUPERGPQA