First Return, Entropy-Eliciting Explore
Reinforcement Learning from Verifiable Rewards (RLVR) improves the reasoning
abilities of Large Language Models (LLMs) but it struggles with unstable
exploration. We propose FR3E (First Return, Entropy-Eliciting Explore), a
structured exploration framework that identifies high-uncertainty decision
points in reasoning trajectories and performs targeted rollouts to construct
semantically grounded intermediate feedback. Our method provides targeted
guidance without relying on dense supervision. Empirical results on
mathematical reasoning benchmarks(AIME24) show that FR3E promotes more stable
training, produces longer and more coherent responses, and increases the
proportion of fully correct trajectories. These results highlight the
framework's effectiveness in improving LLM reasoning through more robust and
structured exploration.
686f39e8d938c25d68441b89
Tianyu Zheng
claimed_verified
2025-07-10T07:09:04.799Z
64ab99dcb76bfd863eba64c1
https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
TY.Zheng
user
aaabiao
686f39e8d938c25d68441b8a
Tianshun Xing
claimed_verified
2025-07-10T09:12:45.801Z
65d2251f98b4a470bf6a26e3
https://cdn-avatars.huggingface.co/v1/production/uploads/65d2251f98b4a470bf6a26e3/C4T0LHYGejrI9mu_k3M8p.jpeg
xts
user
xtsssss
686f39e8d938c25d68441b8b
Qingshui Gu
686f39e8d938c25d68441b8c
Taoran Liang
686f39e8d938c25d68441b8d
Xingwei Qu
686f39e8d938c25d68441b8e
Xin Zhou
686f39e8d938c25d68441b8f
Yizhi Li
claimed_verified
2025-07-25T08:39:27.963Z
6382252f54421460665ec501
https://cdn-avatars.huggingface.co/v1/production/uploads/6382252f54421460665ec501/gW9fev3T5QPcNq4f9hqB1.jpeg
Yizhi Li
user
yizhilll
686f39e8d938c25d68441b90
Zhoufutu Wen
686f39e8d938c25d68441b91
Chenghua Lin
686f39e8d938c25d68441b92
Wenhao Huang
686f39e8d938c25d68441b93
Qian Liu
claimed_verified
2025-07-10T13:14:27.307Z
612ee6a7b960e78c6d2319d4
https://cdn-avatars.huggingface.co/v1/production/uploads/612ee6a7b960e78c6d2319d4/2Hu9BaAyXbyh1vt0v1Qui.jpeg
Qian Liu
user
SivilTaram
686f39e8d938c25d68441b94
Ge Zhang
claimed_verified
2025-07-22T14:00:59.161Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
686f39e8d938c25d68441b95
Zejun Ma
First Return, Entropy-Eliciting Explore
Reinforcement Learning from Verifiable Rewards (RLVR) improves the reasoning
abilities of Large Language Models (LLMs) but it struggles with unstable
exploration. We propose FR3E (First Return, Entropy-Eliciting Explore), a
structured exploration framework that identifies high-uncertainty decision
points in reasoning trajectories and performs targeted rollouts to construct
semantically grounded intermediate feedback. Our method provides targeted
guidance without relying on dense supervision. Empirical results on
mathematical reasoning benchmarks(AIME24) show that FR3E promotes more stable
training, produces longer and more coherent responses, and increases the
proportion of fully correct trajectories. These results highlight the
framework's effectiveness in improving LLM reasoning through more robust and
structured exploration.
FR3E, a structured exploration framework, enhances LLM reasoning by providing targeted guidance at high-uncertainty decision points, leading to more stable training and accurate responses.
Reinforcement Learning from Verifiable Rewards Large Language Models FR3E structured exploration high-uncertainty decision points targeted rollouts semantically grounded intermediate feedback mathematical reasoning benchmarks AIME24