title: Towards System 2 Reasoning in LLMs: Learning How to Think With Meta
    Chain-of-Though
summary: We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends
  traditional Chain-of-Thought (CoT) by explicitly modeling the underlying
  reasoning required to arrive at a particular CoT. We present empirical evidence
  from state-of-the-art models exhibiting behaviors consistent with in-context
  search, and explore methods for producing Meta-CoT via process supervision,
  synthetic data generation, and search algorithms. Finally, we outline a
  concrete pipeline for training a model to produce Meta-CoTs, incorporating
  instruction tuning with linearized search traces and reinforcement learning
  post-training. Finally, we discuss open research questions, including scaling
  laws, verifier roles, and the potential for discovering novel reasoning
  algorithms. This work provides a theoretical and practical roadmap to enable
  Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in
  artificial intelligence.
paper_authors._id: 677f4e0d05ab3422540b88ff
paper_authors.name: Violet Xiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:18.274Z
paper_authors._id: 63a4c25f769ff94bc94ec301
paper_authors.avatarUrl: /avatars/60fd588b972eecca9d74636a244047a1.svg
paper_authors.fullname: Violet Xiang
paper_authors.type: user
paper_authors.user: violetxi
paper_authors._id: 677f4e0d05ab3422540b8900
paper_authors.name: Charlie Snell
paper_authors._id: 677f4e0d05ab3422540b8901
paper_authors.name: Kanishk Gandhi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-10T08:21:32.915Z
paper_authors._id: 63e6a880f2e9a8f22c5a1630
paper_authors.avatarUrl: /avatars/53b57690fe052ce6882bbfc87b11567c.svg
paper_authors.fullname: Kanishk Gandhi
paper_authors.type: user
paper_authors.user: obiwan96
paper_authors._id: 677f4e0d05ab3422540b8902
paper_authors.name: Alon Albalak
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-09T19:57:57.972Z
paper_authors._id: 611a7ec4289467cafea62d13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/611a7ec4289467cafea62d13/pck-0fmPQkoU7yzh6-WoL.jpeg
paper_authors.fullname: Alon Albalak
paper_authors.type: user
paper_authors.user: alon-albalak
paper_authors._id: 677f4e0d05ab3422540b8903
paper_authors.name: Anikait Singh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-09T10:06:59.745Z
paper_authors._id: 6511ee845b7e52b0251fdee9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6511ee845b7e52b0251fdee9/hTIwiIYBGOVnIrxtpri83.png
paper_authors.fullname: Anikait Singh
paper_authors.type: user
paper_authors.user: Asap7772
paper_authors._id: 677f4e0d05ab3422540b8904
paper_authors.name: Chase Blagden
paper_authors._id: 677f4e0d05ab3422540b8905
paper_authors.name: Duy Phung
paper_authors._id: 677f4e0d05ab3422540b8906
paper_authors.name: Rafael Rafailov
paper_authors._id: 677f4e0d05ab3422540b8907
paper_authors.name: Nathan Lile
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-09T10:07:02.418Z
paper_authors._id: 61aa15fd8a9625ebfe284286
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61aa15fd8a9625ebfe284286/KaGzIeijcgcN15JErCqft.jpeg
paper_authors.fullname: nathan lile
paper_authors.type: user
paper_authors.user: nlile
paper_authors._id: 677f4e0d05ab3422540b8908
paper_authors.name: Dakota Mahan
paper_authors._id: 677f4e0d05ab3422540b8909
paper_authors.name: Louis Castricato
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-13T08:54:36.509Z
paper_authors._id: 60d35a0307da9c17c7270909
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665155571839-60d35a0307da9c17c7270909.jpeg
paper_authors.fullname: Louis Castricato
paper_authors.type: user
paper_authors.user: LouisCastricato
paper_authors._id: 677f4e0d05ab3422540b890a
paper_authors.name: Jan-Philipp Franken
paper_authors._id: 677f4e0d05ab3422540b890b
paper_authors.name: Nick Haber
paper_authors._id: 677f4e0d05ab3422540b890c
paper_authors.name: Chelsea Finn
paper_title: Towards System 2 Reasoning in LLMs: Learning How to Think With Meta
    Chain-of-Though
paper_summary: We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends
  traditional Chain-of-Thought (CoT) by explicitly modeling the underlying
  reasoning required to arrive at a particular CoT. We present empirical evidence
  from state-of-the-art models exhibiting behaviors consistent with in-context
  search, and explore methods for producing Meta-CoT via process supervision,
  synthetic data generation, and search algorithms. Finally, we outline a
  concrete pipeline for training a model to produce Meta-CoTs, incorporating
  instruction tuning with linearized search traces and reinforcement learning
  post-training. Finally, we discuss open research questions, including scaling
  laws, verifier roles, and the potential for discovering novel reasoning
  algorithms. This work provides a theoretical and practical roadmap to enable
  Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in
  artificial intelligence.
paper_ai_keywords: Meta Chain-of-Thought (Meta-CoT) Chain-of-Thought (CoT) in-context search process supervision synthetic data generation search algorithms instruction tuning linearized search traces reinforcement learning scaling laws verifier roles reasoning algorithms Latent Chain-of-Thought (LLMs)