title: Genius: A Generalizable and Purely Unsupervised Self-Training Framework
    For Advanced Reasoning
summary: Advancing LLM reasoning skills has captivated wide interest. However, current
  post-training techniques rely heavily on supervisory signals, such as outcome
  supervision or auxiliary reward models, which face the problem of scalability
  and high annotation costs. This motivates us to enhance LLM reasoning without
  the need for external supervision. We introduce a generalizable and purely
  unsupervised self-training framework, named Genius. Without external auxiliary,
  Genius requires to seek the optimal response sequence in a stepwise manner and
  optimize the LLM. To explore the potential steps and exploit the optimal ones,
  Genius introduces a stepwise foresight re-sampling strategy to sample and
  estimate the step value by simulating future outcomes. Further, we recognize
  that the unsupervised setting inevitably induces the intrinsic noise and
  uncertainty. To provide a robust optimization, we propose an
  advantage-calibrated optimization (ACO) loss function to mitigate estimation
  inconsistencies. Combining these techniques together, Genius provides an
  advanced initial step towards self-improve LLM reasoning with general queries
  and without supervision, revolutionizing reasoning scaling laws given the vast
  availability of general queries. The code will be released at
  https://github.com/xufangzhi/Genius.
paper_authors._id: 67fcb7294a92187863e805ee
paper_authors.name: Fangzhi Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T09:46:16.537Z
paper_authors._id: 64e6cf78ecce34cb442dc889
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
paper_authors.fullname: Fangzhi Xu
paper_authors.type: user
paper_authors.user: xufangzhi
paper_authors._id: 67fcb7294a92187863e805ef
paper_authors.name: Hang Yan
paper_authors._id: 67fcb7294a92187863e805f0
paper_authors.name: Chang Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:37:15.750Z
paper_authors._id: 637f22fd932a61b89aeeea37
paper_authors.avatarUrl: /avatars/342957f8242d4edaf1d58e1274313afe.svg
paper_authors.fullname: Chang Ma
paper_authors.type: user
paper_authors.user: changma
paper_authors._id: 67fcb7294a92187863e805f1
paper_authors.name: Haiteng Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:37:07.114Z
paper_authors._id: 64a7c6e223622f7f189bcbe1
paper_authors.avatarUrl: /avatars/4f13a7ed0d2b8d8dfff7dc650e46450a.svg
paper_authors.fullname: haiteng zhao
paper_authors.type: user
paper_authors.user: haitengzhao
paper_authors._id: 67fcb7294a92187863e805f2
paper_authors.name: Qiushi Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T09:24:08.107Z
paper_authors._id: 6064a0eeb1703ddba0d458b9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1617207525789-noauth.png
paper_authors.fullname: Qiushi
paper_authors.type: user
paper_authors.user: QiushiSun
paper_authors._id: 67fcb7294a92187863e805f3
paper_authors.name: Kanzhi Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:37:00.902Z
paper_authors._id: 63340dbbd92c5842ae71d1e9
paper_authors.avatarUrl: /avatars/3a3182996bd41b526dcbfa8687d91963.svg
paper_authors.fullname: Kanzhi Cheng
paper_authors.type: user
paper_authors.user: cckevinn
paper_authors._id: 67fcb7294a92187863e805f4
paper_authors.name: Junxian He
paper_authors._id: 67fcb7294a92187863e805f5
paper_authors.name: Jun Liu
paper_authors._id: 67fcb7294a92187863e805f6
paper_authors.name: Zhiyong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:36:39.666Z
paper_authors._id: 6280e830e99dccaac4bbfde5
paper_authors.avatarUrl: /avatars/9242b8d2826ce2f79af9bb794bba2b61.svg
paper_authors.fullname: Zhiyong Wu
paper_authors.type: user
paper_authors.user: zy001
paper_title: Genius: A Generalizable and Purely Unsupervised Self-Training Framework
    For Advanced Reasoning
paper_summary: Advancing LLM reasoning skills has captivated wide interest. However, current
  post-training techniques rely heavily on supervisory signals, such as outcome
  supervision or auxiliary reward models, which face the problem of scalability
  and high annotation costs. This motivates us to enhance LLM reasoning without
  the need for external supervision. We introduce a generalizable and purely
  unsupervised self-training framework, named Genius. Without external auxiliary,
  Genius requires to seek the optimal response sequence in a stepwise manner and
  optimize the LLM. To explore the potential steps and exploit the optimal ones,
  Genius introduces a stepwise foresight re-sampling strategy to sample and
  estimate the step value by simulating future outcomes. Further, we recognize
  that the unsupervised setting inevitably induces the intrinsic noise and
  uncertainty. To provide a robust optimization, we propose an
  advantage-calibrated optimization (ACO) loss function to mitigate estimation
  inconsistencies. Combining these techniques together, Genius provides an
  advanced initial step towards self-improve LLM reasoning with general queries
  and without supervision, revolutionizing reasoning scaling laws given the vast
  availability of general queries. The code will be released at
  https://github.com/xufangzhi/Genius.
paper_ai_summary: Genius, an unsupervised self-training framework, enhances LLM reasoning by employing a stepwise foresight re-sampling strategy and advantage-calibrated optimization to optimize responses without external supervision.
paper_ai_keywords: post-training techniques outcome supervision auxiliary reward models unsupervised self-training Genius stepwise foresight re-sampling strategy advantage-calibrated optimization (ACO) reasoning scaling laws