title: Thinker: Learning to Think Fast and Slow
summary: Recent studies show that the reasoning capabilities of Large Language Models
  (LLMs) can be improved by applying Reinforcement Learning (RL) to
  question-answering (QA) tasks in areas such as math and coding. With a long
  context length, LLMs may learn to perform search, as indicated by the
  self-correction behavior observed in DeepSeek R1. However, this search behavior
  is often imprecise and lacks confidence, resulting in long, redundant responses
  and highlighting deficiencies in intuition and verification. Inspired by the
  Dual Process Theory in psychology, we introduce a simple modification to the QA
  task that includes four stages: Fast Thinking, where the LLM must answer within
  a strict token budget; Verification, where the model evaluates its initial
  response; Slow Thinking, where it refines the initial response with more
  deliberation; and Summarization, where it distills the refinement from the
  previous stage into precise steps. Our proposed task improves average accuracy
  from 24.9% to 27.9% for Qwen2.5-1.5B, and from 45.9% to 49.8% for
  DeepSeek-R1-Qwen-1.5B. Notably, for Qwen2.5-1.5B, the Fast Thinking mode alone
  achieves 26.8% accuracy using fewer than 1000 tokens, demonstrating substantial
  inference efficiency gains. These findings suggest that intuition and
  deliberative reasoning are distinct, complementary systems benefiting from
  targeted training.
paper_authors._id: 6837361b4b3a674ac2a2c27a
paper_authors.name: Stephen Chung
paper_authors._id: 6837361b4b3a674ac2a2c27b
paper_authors.name: Wenyu Du
paper_authors._id: 6837361b4b3a674ac2a2c27c
paper_authors.name: Jie Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:03:46.383Z
paper_authors._id: 641a6895fb5ffff5ac79d593
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/vxvwsto3llOEWGqQKGMYx.jpeg
paper_authors.fullname: Jie Fu
paper_authors.type: user
paper_authors.user: bigaidream
paper_title: Thinker: Learning to Think Fast and Slow
paper_summary: Recent studies show that the reasoning capabilities of Large Language Models
  (LLMs) can be improved by applying Reinforcement Learning (RL) to
  question-answering (QA) tasks in areas such as math and coding. With a long
  context length, LLMs may learn to perform search, as indicated by the
  self-correction behavior observed in DeepSeek R1. However, this search behavior
  is often imprecise and lacks confidence, resulting in long, redundant responses
  and highlighting deficiencies in intuition and verification. Inspired by the
  Dual Process Theory in psychology, we introduce a simple modification to the QA
  task that includes four stages: Fast Thinking, where the LLM must answer within
  a strict token budget; Verification, where the model evaluates its initial
  response; Slow Thinking, where it refines the initial response with more
  deliberation; and Summarization, where it distills the refinement from the
  previous stage into precise steps. Our proposed task improves average accuracy
  from 24.9% to 27.9% for Qwen2.5-1.5B, and from 45.9% to 49.8% for
  DeepSeek-R1-Qwen-1.5B. Notably, for Qwen2.5-1.5B, the Fast Thinking mode alone
  achieves 26.8% accuracy using fewer than 1000 tokens, demonstrating substantial
  inference efficiency gains. These findings suggest that intuition and
  deliberative reasoning are distinct, complementary systems benefiting from
  targeted training.
paper_ai_summary: A four-stage QA task modification, inspired by Dual Process Theory, improves the accuracy of LLMs in math and coding by separating intuition and deliberation.
paper_ai_keywords: Reinforcement Learning (RL) Question-Answering (QA) DeepSeek R1 Dual Process Theory Fast Thinking Verification Slow Thinking Summarization