title: General Reasoning Requires Learning to Reason from the Get-go
summary: Large Language Models (LLMs) have demonstrated impressive real-world utility,
  exemplifying artificial useful intelligence (AUI). However, their ability to
  reason adaptively and robustly -- the hallmarks of artificial general
  intelligence (AGI) -- remains fragile. While LLMs seemingly succeed in
  commonsense reasoning, programming, and mathematics, they struggle to
  generalize algorithmic understanding across novel contexts. Our experiments
  with algorithmic tasks in esoteric programming languages reveal that LLM's
  reasoning overfits to the training data and is limited in its transferability.
  We hypothesize that the core issue underlying such limited transferability is
  the coupling of reasoning and knowledge in LLMs.
    To transition from AUI to AGI, we propose disentangling knowledge and
  reasoning through three key directions: (1) pretaining to reason using RL from
  scratch as an alternative to the widely used next-token prediction pretraining,
  (2) using a curriculum of synthetic tasks to ease the learning of a
  reasoning prior for RL that can then be transferred to natural
  language tasks, and (3) learning more generalizable reasoning functions using a
  small context window to reduce exploiting spurious correlations between tokens.
  Such a reasoning system coupled with a trained retrieval system and a large
  external memory bank as a knowledge store can overcome several limitations of
  existing architectures at learning to reason in novel scenarios.
paper_authors._id: 67c66a6321d722b4247e5959
paper_authors.name: Seungwook Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T16:08:58.266Z
paper_authors._id: 6520d6db2a16045c092b3b36
paper_authors.avatarUrl: /avatars/dab34f141a1aef39d00c789ff85e729f.svg
paper_authors.fullname: Seungwook Han
paper_authors.type: user
paper_authors.user: hanseungwook
paper_authors._id: 67c66a6321d722b4247e595a
paper_authors.name: Jyothish Pari
paper_authors._id: 67c66a6321d722b4247e595b
paper_authors.name: Samuel J. Gershman
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-04T13:57:29.748Z
paper_authors._id: 6520d6db2a16045c092b3b36
paper_authors.avatarUrl: /avatars/dab34f141a1aef39d00c789ff85e729f.svg
paper_authors.fullname: Seungwook Han
paper_authors.type: user
paper_authors.user: hanseungwook
paper_authors._id: 67c66a6321d722b4247e595c
paper_authors.name: Pulkit Agrawal
paper_title: General Reasoning Requires Learning to Reason from the Get-go
paper_summary: Large Language Models (LLMs) have demonstrated impressive real-world utility,
  exemplifying artificial useful intelligence (AUI). However, their ability to
  reason adaptively and robustly -- the hallmarks of artificial general
  intelligence (AGI) -- remains fragile. While LLMs seemingly succeed in
  commonsense reasoning, programming, and mathematics, they struggle to
  generalize algorithmic understanding across novel contexts. Our experiments
  with algorithmic tasks in esoteric programming languages reveal that LLM's
  reasoning overfits to the training data and is limited in its transferability.
  We hypothesize that the core issue underlying such limited transferability is
  the coupling of reasoning and knowledge in LLMs.
    To transition from AUI to AGI, we propose disentangling knowledge and
  reasoning through three key directions: (1) pretaining to reason using RL from
  scratch as an alternative to the widely used next-token prediction pretraining,
  (2) using a curriculum of synthetic tasks to ease the learning of a
  reasoning prior for RL that can then be transferred to natural
  language tasks, and (3) learning more generalizable reasoning functions using a
  small context window to reduce exploiting spurious correlations between tokens.
  Such a reasoning system coupled with a trained retrieval system and a large
  external memory bank as a knowledge store can overcome several limitations of
  existing architectures at learning to reason in novel scenarios.
paper_ai_keywords: Large Language Models (LLMs) artificial useful intelligence (AUI) artificial general intelligence (AGI) commonsense reasoning programming mathematics algorithmic tasks esoteric programming languages reasoning overfitting transferability RL (Reinforcement Learning) next-token prediction curriculum of synthetic tasks reasoning prior small context window spurious correlations between tokens retrieval system large external memory bank