title: Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement
    Learning
summary: Inspired by the success of DeepSeek-R1, we explore the potential of
  rule-based reinforcement learning (RL) in large reasoning models. To analyze
  reasoning dynamics, we use synthetic logic puzzles as training data due to
  their controllable complexity and straightforward answer verification. We make
  some key technical contributions that lead to effective and stable RL training:
  a system prompt that emphasizes the thinking and answering process, a stringent
  format reward function that penalizes outputs for taking shortcuts, and a
  straightforward training recipe that achieves stable convergence. Our 7B model
  develops advanced reasoning skills-such as reflection, verification, and
  summarization-that are absent from the logic corpus. Remarkably, after training
  on just 5K logic problems, it demonstrates generalization abilities to the
  challenging math benchmarks AIME and AMC.
paper_authors._id: 67b7f08c357c2729ac20a81b
paper_authors.name: Tian Xie
paper_authors._id: 67b7f08c357c2729ac20a81c
paper_authors.name: Zitian Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:50:06.783Z
paper_authors._id: 641ddac5be3bd3a5a06ed4a4
paper_authors.avatarUrl: /avatars/14969dff861d53b0a75305606495eca7.svg
paper_authors.fullname: zitian gao
paper_authors.type: user
paper_authors.user: zgao3186
paper_authors._id: 67b7f08c357c2729ac20a81d
paper_authors.name: Qingnan Ren
paper_authors._id: 67b7f08c357c2729ac20a81e
paper_authors.name: Haoming Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T09:23:21.219Z
paper_authors._id: 6501535970b6b05c5af84383
paper_authors.avatarUrl: /avatars/a827dfa11589cabd6868c617eeecbbba.svg
paper_authors.fullname: Haoming Luo
paper_authors.type: user
paper_authors.user: Resnet-340
paper_authors._id: 67b7f08c357c2729ac20a81f
paper_authors.name: Yuqian Hong
paper_authors._id: 67b7f08c357c2729ac20a820
paper_authors.name: Bryan Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:49:20.858Z
paper_authors._id: 67090a395f4ed2ff1f0b3658
paper_authors.avatarUrl: /avatars/03744a2fcbcb0e8074c04bad83b3e34c.svg
paper_authors.fullname: Zhenbang Dai
paper_authors.type: user
paper_authors.user: BryanDai
paper_authors._id: 67b7f08c357c2729ac20a821
paper_authors.name: Joey Zhou
paper_authors._id: 67b7f08c357c2729ac20a822
paper_authors.name: Kai Qiu
paper_authors._id: 67b7f08c357c2729ac20a823
paper_authors.name: Zhirong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:48:55.865Z
paper_authors._id: 67034414f2b11c7dd251e232
paper_authors.avatarUrl: /avatars/6b741ac2eab48c6f72185342f9af7d1f.svg
paper_authors.fullname: wzr
paper_authors.type: user
paper_authors.user: wuzhirong
paper_authors._id: 67b7f08c357c2729ac20a824
paper_authors.name: Chong Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:48:44.370Z
paper_authors._id: 676a328148d749b7086782d0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Tt7u8l8f_1oVBWmBp7tkm.png
paper_authors.fullname: Chong Luo
paper_authors.type: user
paper_authors.user: cluo-ms
paper_title: Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement
    Learning
paper_summary: Inspired by the success of DeepSeek-R1, we explore the potential of
  rule-based reinforcement learning (RL) in large reasoning models. To analyze
  reasoning dynamics, we use synthetic logic puzzles as training data due to
  their controllable complexity and straightforward answer verification. We make
  some key technical contributions that lead to effective and stable RL training:
  a system prompt that emphasizes the thinking and answering process, a stringent
  format reward function that penalizes outputs for taking shortcuts, and a
  straightforward training recipe that achieves stable convergence. Our 7B model
  develops advanced reasoning skills-such as reflection, verification, and
  summarization-that are absent from the logic corpus. Remarkably, after training
  on just 5K logic problems, it demonstrates generalization abilities to the
  challenging math benchmarks AIME and AMC.
paper_ai_keywords: rule-based reinforcement learning (RL) Transformers synthesis logic puzzles reasoning dynamics system prompt format reward function self-risk mitigation reflection verification summarization rigid bodies generalization abilities math benchmarks AIME AMC