Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient
  in Latent Space
Reasoning ability, a core component of human intelligence, continues to pose
a significant challenge for Large Language Models (LLMs) in the pursuit of AGI.
Although model performance has improved under the training scaling law,
significant challenges remain, particularly with respect to training
algorithms, such as catastrophic forgetting, and the limited availability of
novel training data. As an alternative, test-time scaling enhances reasoning
performance by increasing test-time computation without parameter updating.
Unlike prior methods in this paradigm focused on token space, we propose
leveraging latent space for more effective reasoning and better adherence to
the test-time scaling law. We introduce LatentSeek, a novel framework that
enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA)
within the model's latent space. Specifically, LatentSeek leverages policy
gradient to iteratively update latent representations, guided by self-generated
reward signals. LatentSeek is evaluated on a range of reasoning benchmarks,
including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures.
Results show that LatentSeek consistently outperforms strong baselines, such as
Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our
analysis demonstrates that LatentSeek is highly efficient, typically converging
within a few iterations for problems of average complexity, while also
benefiting from additional iterations, thereby highlighting the potential of
test-time scaling in the latent space. These findings position LatentSeek as a
lightweight, scalable, and effective solution for enhancing the reasoning
capabilities of LLMs.
682c154830991f1cf6291a79
Hengli Li
claimed_verified
2025-05-20T07:19:59.597Z
62649e2b1ed8d81e47ad9b4e
/avatars/f33a0b727822fd2ea99dce37fbda3d17.svg
Li
user
henry12348
682c154830991f1cf6291a7a
Chenxi Li
682c154830991f1cf6291a7b
Tong Wu
682c154830991f1cf6291a7c
Xuekai Zhu
admin_assigned
2025-05-20T08:29:34.137Z
647ffddeb82adfa7cc1a10d9
/avatars/26aa168d6b2068298ebb16584aa52b6c.svg
zhu
user
xuekai
682c154830991f1cf6291a7d
Yuxuan Wang
admin_assigned
2025-05-20T08:29:49.123Z
60b9e6837946aff342f734ae
/avatars/a711a6aa35757dfd7b78b26098a964fc.svg
Yuxuan Wang
user
ColorfulAI
682c154830991f1cf6291a7e
Zhaoxin Yu
682c154830991f1cf6291a7f
Eric Hanchen Jiang
682c154830991f1cf6291a80
Song-Chun Zhu
682c154830991f1cf6291a81
Zixia Jia
admin_assigned
2025-05-20T08:30:26.630Z
64b7ae6cf53ae848e72b997d
/avatars/b55dd3d6fcb3ccac2e3880d01a9bdc63.svg
Zixia Jia
user
vickyandkekey
682c154830991f1cf6291a82
Ying Nian Wu
682c154830991f1cf6291a83
Zilong Zheng
extracted_pending
2025-05-20T05:38:17.771Z
63a95a6a7930fa8c7dd63d4e
/avatars/d9d0420f7ddfe2f3a7e029fb05f1c89f.svg
Zilong Zheng
user
zlzheng
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient
  in Latent Space
Reasoning ability, a core component of human intelligence, continues to pose
a significant challenge for Large Language Models (LLMs) in the pursuit of AGI.
Although model performance has improved under the training scaling law,
significant challenges remain, particularly with respect to training
algorithms, such as catastrophic forgetting, and the limited availability of
novel training data. As an alternative, test-time scaling enhances reasoning
performance by increasing test-time computation without parameter updating.
Unlike prior methods in this paradigm focused on token space, we propose
leveraging latent space for more effective reasoning and better adherence to
the test-time scaling law. We introduce LatentSeek, a novel framework that
enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA)
within the model's latent space. Specifically, LatentSeek leverages policy
gradient to iteratively update latent representations, guided by self-generated
reward signals. LatentSeek is evaluated on a range of reasoning benchmarks,
including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures.
Results show that LatentSeek consistently outperforms strong baselines, such as
Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our
analysis demonstrates that LatentSeek is highly efficient, typically converging
within a few iterations for problems of average complexity, while also
benefiting from additional iterations, thereby highlighting the potential of
test-time scaling in the latent space. These findings position LatentSeek as a
lightweight, scalable, and effective solution for enhancing the reasoning
capabilities of LLMs.
LatentSeek enhances LLM reasoning using test-time instance-level adaptation in latent space, improving performance across various benchmarks.
Large Language Models (LLMs) AGI catastrophic forgetting token space latent space LatentSeek Test-Time Instance-level Adaptation (TTIA) policy gradient self-generated reward signals GSM8K MATH-500 AIME2024 Chain-of-Thought prompting