title: Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient
    in Latent Space
summary: Reasoning ability, a core component of human intelligence, continues to pose
  a significant challenge for Large Language Models (LLMs) in the pursuit of AGI.
  Although model performance has improved under the training scaling law,
  significant challenges remain, particularly with respect to training
  algorithms, such as catastrophic forgetting, and the limited availability of
  novel training data. As an alternative, test-time scaling enhances reasoning
  performance by increasing test-time computation without parameter updating.
  Unlike prior methods in this paradigm focused on token space, we propose
  leveraging latent space for more effective reasoning and better adherence to
  the test-time scaling law. We introduce LatentSeek, a novel framework that
  enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA)
  within the model's latent space. Specifically, LatentSeek leverages policy
  gradient to iteratively update latent representations, guided by self-generated
  reward signals. LatentSeek is evaluated on a range of reasoning benchmarks,
  including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures.
  Results show that LatentSeek consistently outperforms strong baselines, such as
  Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our
  analysis demonstrates that LatentSeek is highly efficient, typically converging
  within a few iterations for problems of average complexity, while also
  benefiting from additional iterations, thereby highlighting the potential of
  test-time scaling in the latent space. These findings position LatentSeek as a
  lightweight, scalable, and effective solution for enhancing the reasoning
  capabilities of LLMs.
paper_authors._id: 682c154830991f1cf6291a79
paper_authors.name: Hengli Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:19:59.597Z
paper_authors._id: 62649e2b1ed8d81e47ad9b4e
paper_authors.avatarUrl: /avatars/f33a0b727822fd2ea99dce37fbda3d17.svg
paper_authors.fullname: Li
paper_authors.type: user
paper_authors.user: henry12348
paper_authors._id: 682c154830991f1cf6291a7a
paper_authors.name: Chenxi Li
paper_authors._id: 682c154830991f1cf6291a7b
paper_authors.name: Tong Wu
paper_authors._id: 682c154830991f1cf6291a7c
paper_authors.name: Xuekai Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:29:34.137Z
paper_authors._id: 647ffddeb82adfa7cc1a10d9
paper_authors.avatarUrl: /avatars/26aa168d6b2068298ebb16584aa52b6c.svg
paper_authors.fullname: zhu
paper_authors.type: user
paper_authors.user: xuekai
paper_authors._id: 682c154830991f1cf6291a7d
paper_authors.name: Yuxuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:29:49.123Z
paper_authors._id: 60b9e6837946aff342f734ae
paper_authors.avatarUrl: /avatars/a711a6aa35757dfd7b78b26098a964fc.svg
paper_authors.fullname: Yuxuan Wang
paper_authors.type: user
paper_authors.user: ColorfulAI
paper_authors._id: 682c154830991f1cf6291a7e
paper_authors.name: Zhaoxin Yu
paper_authors._id: 682c154830991f1cf6291a7f
paper_authors.name: Eric Hanchen Jiang
paper_authors._id: 682c154830991f1cf6291a80
paper_authors.name: Song-Chun Zhu
paper_authors._id: 682c154830991f1cf6291a81
paper_authors.name: Zixia Jia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:30:26.630Z
paper_authors._id: 64b7ae6cf53ae848e72b997d
paper_authors.avatarUrl: /avatars/b55dd3d6fcb3ccac2e3880d01a9bdc63.svg
paper_authors.fullname: Zixia Jia
paper_authors.type: user
paper_authors.user: vickyandkekey
paper_authors._id: 682c154830991f1cf6291a82
paper_authors.name: Ying Nian Wu
paper_authors._id: 682c154830991f1cf6291a83
paper_authors.name: Zilong Zheng
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-20T05:38:17.771Z
paper_authors._id: 63a95a6a7930fa8c7dd63d4e
paper_authors.avatarUrl: /avatars/d9d0420f7ddfe2f3a7e029fb05f1c89f.svg
paper_authors.fullname: Zilong Zheng
paper_authors.type: user
paper_authors.user: zlzheng
paper_title: Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient
    in Latent Space
paper_summary: Reasoning ability, a core component of human intelligence, continues to pose
  a significant challenge for Large Language Models (LLMs) in the pursuit of AGI.
  Although model performance has improved under the training scaling law,
  significant challenges remain, particularly with respect to training
  algorithms, such as catastrophic forgetting, and the limited availability of
  novel training data. As an alternative, test-time scaling enhances reasoning
  performance by increasing test-time computation without parameter updating.
  Unlike prior methods in this paradigm focused on token space, we propose
  leveraging latent space for more effective reasoning and better adherence to
  the test-time scaling law. We introduce LatentSeek, a novel framework that
  enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA)
  within the model's latent space. Specifically, LatentSeek leverages policy
  gradient to iteratively update latent representations, guided by self-generated
  reward signals. LatentSeek is evaluated on a range of reasoning benchmarks,
  including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures.
  Results show that LatentSeek consistently outperforms strong baselines, such as
  Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our
  analysis demonstrates that LatentSeek is highly efficient, typically converging
  within a few iterations for problems of average complexity, while also
  benefiting from additional iterations, thereby highlighting the potential of
  test-time scaling in the latent space. These findings position LatentSeek as a
  lightweight, scalable, and effective solution for enhancing the reasoning
  capabilities of LLMs.
paper_ai_summary: LatentSeek enhances LLM reasoning using test-time instance-level adaptation in latent space, improving performance across various benchmarks.
paper_ai_keywords: Large Language Models (LLMs) AGI catastrophic forgetting token space latent space LatentSeek Test-Time Instance-level Adaptation (TTIA) policy gradient self-generated reward signals GSM8K MATH-500 AIME2024 Chain-of-Thought prompting