title: Z1: Efficient Test-time Scaling with Code
summary: Large Language Models (LLMs) can achieve enhanced complex problem-solving
  through test-time computing scaling, yet this often entails longer contexts and
  numerous reasoning token costs. In this paper, we propose an efficient
  test-time scaling method that trains LLMs on code-related reasoning
  trajectories, facilitating their reduction of excess thinking tokens while
  maintaining performance. First, we create Z1-Code-Reasoning-107K, a curated
  dataset of simple and complex coding problems paired with their short and long
  solution trajectories. Second, we present a novel Shifted Thinking Window to
  mitigate overthinking overhead by removing context-delimiting tags (e.g.,
  <think>. . . </think>) and capping reasoning tokens. Trained with long and
  short trajectory data and equipped with Shifted Thinking Window, our model,
  Z1-7B, demonstrates the ability to adjust its reasoning level as the complexity
  of problems and exhibits efficient test-time scaling across different reasoning
  tasks that matches R1-Distill-Qwen-7B performance with about 30% of its average
  thinking tokens. Notably, fine-tuned with only code trajectories, Z1-7B
  demonstrates generalization to broader reasoning tasks (47.5% on GPQA Diamond).
  Our analysis of efficient reasoning elicitation also provides valuable insights
  for future research.
paper_authors._id: 67eca0c4cfce948cbbbcff9a
paper_authors.name: Zhaojian Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:38.129Z
paper_authors._id: 646f3443c261dc413383b8a4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646f3443c261dc413383b8a4/7R2XX3VqQ6DmgQoMU7Un4.jpeg
paper_authors.fullname: Zhaojian Yu
paper_authors.type: user
paper_authors.user: zjy2001
paper_authors._id: 67eca0c4cfce948cbbbcff9b
paper_authors.name: Yinghao Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:50:27.577Z
paper_authors._id: 650ed552dc509ae7d7bb1ccc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650ed552dc509ae7d7bb1ccc/CPR6gaIjPfnBGfr-D_rpO.jpeg
paper_authors.fullname: Yinghao Wu
paper_authors.type: user
paper_authors.user: yh1567
paper_authors._id: 67eca0c4cfce948cbbbcff9c
paper_authors.name: Yilun Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:34.689Z
paper_authors._id: 62f662bcc58915315c4eccea
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f662bcc58915315c4eccea/zOAQLONfMP88zr70sxHK-.jpeg
paper_authors.fullname: Yilun Zhao
paper_authors.type: user
paper_authors.user: yilunzhao
paper_authors._id: 67eca0c4cfce948cbbbcff9d
paper_authors.name: Arman Cohan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:50:33.530Z
paper_authors._id: 5f5ba21188f57f65f951f255
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
paper_authors.fullname: Arman Cohan
paper_authors.type: user
paper_authors.user: armanc
paper_authors._id: 67eca0c4cfce948cbbbcff9e
paper_authors.name: Xiao-Ping Zhang
paper_title: Z1: Efficient Test-time Scaling with Code
paper_summary: Large Language Models (LLMs) can achieve enhanced complex problem-solving
  through test-time computing scaling, yet this often entails longer contexts and
  numerous reasoning token costs. In this paper, we propose an efficient
  test-time scaling method that trains LLMs on code-related reasoning
  trajectories, facilitating their reduction of excess thinking tokens while
  maintaining performance. First, we create Z1-Code-Reasoning-107K, a curated
  dataset of simple and complex coding problems paired with their short and long
  solution trajectories. Second, we present a novel Shifted Thinking Window to
  mitigate overthinking overhead by removing context-delimiting tags (e.g.,
  <think>. . . </think>) and capping reasoning tokens. Trained with long and
  short trajectory data and equipped with Shifted Thinking Window, our model,
  Z1-7B, demonstrates the ability to adjust its reasoning level as the complexity
  of problems and exhibits efficient test-time scaling across different reasoning
  tasks that matches R1-Distill-Qwen-7B performance with about 30% of its average
  thinking tokens. Notably, fine-tuned with only code trajectories, Z1-7B
  demonstrates generalization to broader reasoning tasks (47.5% on GPQA Diamond).
  Our analysis of efficient reasoning elicitation also provides valuable insights
  for future research.
paper_ai_summary: A novel Shifted Thinking Window method trains LLMs on code-related reasoning trajectories to reduce excessive thinking tokens while maintaining performance and efficient test-time scaling.
paper_ai_keywords: LLMs test-time computing scaling reasoning trajectories Z1-Code-Reasoning-107K Shifted Thinking Window context-delimiting tags reasoning tokens Z1-7B efficient test-time scaling R1-Distill-Qwen-7B fine-tuning GPQA Diamond efficient reasoning elicitation