title: Learning from Peers in Reasoning Models
summary: Large Reasoning Models (LRMs) have the ability to self-correct even when they
  make mistakes in their reasoning paths. However, our study reveals that when
  the reasoning process starts with a short but poor beginning, it becomes
  difficult for the model to recover. We refer to this phenomenon as the "Prefix
  Dominance Trap". Inspired by psychological findings that peer interaction can
  promote self-correction without negatively impacting already accurate
  individuals, we propose **Learning from Peers** (LeaP) to address this
  phenomenon. Specifically, every tokens, each reasoning path summarizes its
  intermediate reasoning and shares it with others through a routing mechanism,
  enabling paths to incorporate peer insights during inference. However, we
  observe that smaller models sometimes fail to follow summarization and
  reflection instructions effectively. To address this, we fine-tune them into
  our **LeaP-T** model series. Experiments on AIME 2024, AIME 2025, AIMO 2025,
  and GPQA Diamond show that LeaP provides substantial improvements. For
  instance, QwQ-32B with LeaP achieves nearly 5 absolute points higher than the
  baseline on average, and surpasses DeepSeek-R1-671B on three math benchmarks
  with an average gain of 3.3 points. Notably, our fine-tuned LeaP-T-7B matches
  the performance of DeepSeek-R1-Distill-Qwen-14B on AIME 2024. In-depth analysis
  reveals LeaP's robust error correction by timely peer insights, showing strong
  error tolerance and handling varied task difficulty. LeaP marks a milestone by
  enabling LRMs to collaborate during reasoning. Our code, datasets, and models
  are available at https://learning-from-peers.github.io/ .
paper_authors._id: 6822a9b0e6328d29a9c721d6
paper_authors.name: Tongxu Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:12:19.861Z
paper_authors._id: 6421b07e918f0fd889f0a682
paper_authors.avatarUrl: /avatars/314b55c2428426c846d9449f98db4355.svg
paper_authors.fullname: Tongxu Luo
paper_authors.type: user
paper_authors.user: Zeno-Luo
paper_authors._id: 6822a9b0e6328d29a9c721d7
paper_authors.name: Wenyu Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:30:44.940Z
paper_authors._id: 624c3d2ca19f20b197761ba9
paper_authors.avatarUrl: /avatars/7a64b81c29f4f6700fa18effc5616865.svg
paper_authors.fullname: Wenyu Du
paper_authors.type: user
paper_authors.user: wydu
paper_authors._id: 6822a9b0e6328d29a9c721d8
paper_authors.name: Jiaxi Bi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:12:14.622Z
paper_authors._id: 6819c4b3ba781b9ebb08cf48
paper_authors.avatarUrl: /avatars/6a51be63df7469d34a2f5c5229dac88d.svg
paper_authors.fullname: Jiaxi Bi
paper_authors.type: user
paper_authors.user: Jiaxi0775
paper_authors._id: 6822a9b0e6328d29a9c721d9
paper_authors.name: Stephen Chung
paper_authors._id: 6822a9b0e6328d29a9c721da
paper_authors.name: Zhengyang Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:31:01.937Z
paper_authors._id: 64912976b95c3f0a1e6233cb
paper_authors.avatarUrl: /avatars/c0615f8c6606073faffb419757d4e667.svg
paper_authors.fullname: Zhengyang Tang
paper_authors.type: user
paper_authors.user: tangzhy
paper_authors._id: 6822a9b0e6328d29a9c721db
paper_authors.name: Hao Yang
paper_authors._id: 6822a9b0e6328d29a9c721dc
paper_authors.name: Min Zhang
paper_authors._id: 6822a9b0e6328d29a9c721dd
paper_authors.name: Benyou Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:31:21.723Z
paper_authors._id: 637c6703ca8542a0ba900ccb
paper_authors.avatarUrl: /avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg
paper_authors.fullname: Wang
paper_authors.type: user
paper_authors.user: Benyou
paper_title: Learning from Peers in Reasoning Models
paper_summary: Large Reasoning Models (LRMs) have the ability to self-correct even when they
  make mistakes in their reasoning paths. However, our study reveals that when
  the reasoning process starts with a short but poor beginning, it becomes
  difficult for the model to recover. We refer to this phenomenon as the "Prefix
  Dominance Trap". Inspired by psychological findings that peer interaction can
  promote self-correction without negatively impacting already accurate
  individuals, we propose **Learning from Peers** (LeaP) to address this
  phenomenon. Specifically, every tokens, each reasoning path summarizes its
  intermediate reasoning and shares it with others through a routing mechanism,
  enabling paths to incorporate peer insights during inference. However, we
  observe that smaller models sometimes fail to follow summarization and
  reflection instructions effectively. To address this, we fine-tune them into
  our **LeaP-T** model series. Experiments on AIME 2024, AIME 2025, AIMO 2025,
  and GPQA Diamond show that LeaP provides substantial improvements. For
  instance, QwQ-32B with LeaP achieves nearly 5 absolute points higher than the
  baseline on average, and surpasses DeepSeek-R1-671B on three math benchmarks
  with an average gain of 3.3 points. Notably, our fine-tuned LeaP-T-7B matches
  the performance of DeepSeek-R1-Distill-Qwen-14B on AIME 2024. In-depth analysis
  reveals LeaP's robust error correction by timely peer insights, showing strong
  error tolerance and handling varied task difficulty. LeaP marks a milestone by
  enabling LRMs to collaborate during reasoning. Our code, datasets, and models
  are available at https://learning-from-peers.github.io/ .
paper_ai_summary: LeaP, a peer interaction mechanism for large reasoning models, improves error correction and performance across various math benchmarks by enabling collaborative reasoning paths.
paper_ai_keywords: Learning from Peers (LeaP) Prefix Dominance Trap peer interaction routing mechanism intermediate reasoning fine-tuning LeaP-T error correction QwQ-32B DeepSeek-R1-671B DeepSeek-R1-Distill-Qwen-14B AIME 2024 AIME 2025 AIMO 2025 GPQA Diamond