Genius: A Generalizable and Purely Unsupervised Self-Training Framework
  For Advanced Reasoning
Advancing LLM reasoning skills has captivated wide interest. However, current
post-training techniques rely heavily on supervisory signals, such as outcome
supervision or auxiliary reward models, which face the problem of scalability
and high annotation costs. This motivates us to enhance LLM reasoning without
the need for external supervision. We introduce a generalizable and purely
unsupervised self-training framework, named Genius. Without external auxiliary,
Genius requires to seek the optimal response sequence in a stepwise manner and
optimize the LLM. To explore the potential steps and exploit the optimal ones,
Genius introduces a stepwise foresight re-sampling strategy to sample and
estimate the step value by simulating future outcomes. Further, we recognize
that the unsupervised setting inevitably induces the intrinsic noise and
uncertainty. To provide a robust optimization, we propose an
advantage-calibrated optimization (ACO) loss function to mitigate estimation
inconsistencies. Combining these techniques together, Genius provides an
advanced initial step towards self-improve LLM reasoning with general queries
and without supervision, revolutionizing reasoning scaling laws given the vast
availability of general queries. The code will be released at
https://github.com/xufangzhi/Genius.
67fcb7294a92187863e805ee
Fangzhi Xu
claimed_verified
2025-04-14T09:46:16.537Z
64e6cf78ecce34cb442dc889
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
Fangzhi Xu
user
xufangzhi
67fcb7294a92187863e805ef
Hang Yan
67fcb7294a92187863e805f0
Chang Ma
admin_assigned
2025-04-17T08:37:15.750Z
637f22fd932a61b89aeeea37
/avatars/342957f8242d4edaf1d58e1274313afe.svg
Chang Ma
user
changma
67fcb7294a92187863e805f1
Haiteng Zhao
admin_assigned
2025-04-17T08:37:07.114Z
64a7c6e223622f7f189bcbe1
/avatars/4f13a7ed0d2b8d8dfff7dc650e46450a.svg
haiteng zhao
user
haitengzhao
67fcb7294a92187863e805f2
Qiushi Sun
claimed_verified
2025-04-16T09:24:08.107Z
6064a0eeb1703ddba0d458b9
https://cdn-avatars.huggingface.co/v1/production/uploads/1617207525789-noauth.png
Qiushi
user
QiushiSun
67fcb7294a92187863e805f3
Kanzhi Cheng
admin_assigned
2025-04-17T08:37:00.902Z
63340dbbd92c5842ae71d1e9
/avatars/3a3182996bd41b526dcbfa8687d91963.svg
Kanzhi Cheng
user
cckevinn
67fcb7294a92187863e805f4
Junxian He
67fcb7294a92187863e805f5
Jun Liu
67fcb7294a92187863e805f6
Zhiyong Wu
admin_assigned
2025-04-17T08:36:39.666Z
6280e830e99dccaac4bbfde5
/avatars/9242b8d2826ce2f79af9bb794bba2b61.svg
Zhiyong Wu
user
zy001
Genius: A Generalizable and Purely Unsupervised Self-Training Framework
  For Advanced Reasoning
Advancing LLM reasoning skills has captivated wide interest. However, current
post-training techniques rely heavily on supervisory signals, such as outcome
supervision or auxiliary reward models, which face the problem of scalability
and high annotation costs. This motivates us to enhance LLM reasoning without
the need for external supervision. We introduce a generalizable and purely
unsupervised self-training framework, named Genius. Without external auxiliary,
Genius requires to seek the optimal response sequence in a stepwise manner and
optimize the LLM. To explore the potential steps and exploit the optimal ones,
Genius introduces a stepwise foresight re-sampling strategy to sample and
estimate the step value by simulating future outcomes. Further, we recognize
that the unsupervised setting inevitably induces the intrinsic noise and
uncertainty. To provide a robust optimization, we propose an
advantage-calibrated optimization (ACO) loss function to mitigate estimation
inconsistencies. Combining these techniques together, Genius provides an
advanced initial step towards self-improve LLM reasoning with general queries
and without supervision, revolutionizing reasoning scaling laws given the vast
availability of general queries. The code will be released at
https://github.com/xufangzhi/Genius.
Genius, an unsupervised self-training framework, enhances LLM reasoning by employing a stepwise foresight re-sampling strategy and advantage-calibrated optimization to optimize responses without external supervision.
post-training techniques outcome supervision auxiliary reward models unsupervised self-training Genius stepwise foresight re-sampling strategy advantage-calibrated optimization (ACO) reasoning scaling laws