Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement
  Learning
Inspired by the success of DeepSeek-R1, we explore the potential of
rule-based reinforcement learning (RL) in large reasoning models. To analyze
reasoning dynamics, we use synthetic logic puzzles as training data due to
their controllable complexity and straightforward answer verification. We make
some key technical contributions that lead to effective and stable RL training:
a system prompt that emphasizes the thinking and answering process, a stringent
format reward function that penalizes outputs for taking shortcuts, and a
straightforward training recipe that achieves stable convergence. Our 7B model
develops advanced reasoning skills-such as reflection, verification, and
summarization-that are absent from the logic corpus. Remarkably, after training
on just 5K logic problems, it demonstrates generalization abilities to the
challenging math benchmarks AIME and AMC.
67b7f08c357c2729ac20a81b
Tian Xie
67b7f08c357c2729ac20a81c
Zitian Gao
admin_assigned
2025-02-21T14:50:06.783Z
641ddac5be3bd3a5a06ed4a4
/avatars/14969dff861d53b0a75305606495eca7.svg
zitian gao
user
zgao3186
67b7f08c357c2729ac20a81d
Qingnan Ren
67b7f08c357c2729ac20a81e
Haoming Luo
claimed_verified
2025-02-24T09:23:21.219Z
6501535970b6b05c5af84383
/avatars/a827dfa11589cabd6868c617eeecbbba.svg
Haoming Luo
user
Resnet-340
67b7f08c357c2729ac20a81f
Yuqian Hong
67b7f08c357c2729ac20a820
Bryan Dai
admin_assigned
2025-02-21T14:49:20.858Z
67090a395f4ed2ff1f0b3658
/avatars/03744a2fcbcb0e8074c04bad83b3e34c.svg
Zhenbang Dai
user
BryanDai
67b7f08c357c2729ac20a821
Joey Zhou
67b7f08c357c2729ac20a822
Kai Qiu
67b7f08c357c2729ac20a823
Zhirong Wu
admin_assigned
2025-02-21T14:48:55.865Z
67034414f2b11c7dd251e232
/avatars/6b741ac2eab48c6f72185342f9af7d1f.svg
wzr
user
wuzhirong
67b7f08c357c2729ac20a824
Chong Luo
admin_assigned
2025-02-21T14:48:44.370Z
676a328148d749b7086782d0
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Tt7u8l8f_1oVBWmBp7tkm.png
Chong Luo
user
cluo-ms
Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement
  Learning
Inspired by the success of DeepSeek-R1, we explore the potential of
rule-based reinforcement learning (RL) in large reasoning models. To analyze
reasoning dynamics, we use synthetic logic puzzles as training data due to
their controllable complexity and straightforward answer verification. We make
some key technical contributions that lead to effective and stable RL training:
a system prompt that emphasizes the thinking and answering process, a stringent
format reward function that penalizes outputs for taking shortcuts, and a
straightforward training recipe that achieves stable convergence. Our 7B model
develops advanced reasoning skills-such as reflection, verification, and
summarization-that are absent from the logic corpus. Remarkably, after training
on just 5K logic problems, it demonstrates generalization abilities to the
challenging math benchmarks AIME and AMC.
A rule-based reinforcement learning system for large reasoning models uses synthetic logic puzzles to develop advanced reasoning skills and demonstrates generalization on complex math benchmarks.
rule-based reinforcement learning RL synthetic logic puzzles format reward function stable convergence reflection verification summarization math benchmarks AIME AMC