title: On the Diagram of Thought
summary: We introduce Diagram of Thought (DoT), a framework that models iterative
  reasoning in large language models (LLMs) as the construction of a directed
  acyclic graph (DAG) within a single model. Unlike traditional approaches that
  represent reasoning as linear chains or trees, DoT organizes propositions,
  critiques, refinements, and verifications into a cohesive DAG structure,
  allowing the model to explore complex reasoning pathways while maintaining
  logical consistency. Each node in the diagram corresponds to a proposition that
  has been proposed, critiqued, refined, or verified, enabling the LLM to
  iteratively improve its reasoning through natural language feedback. By
  leveraging auto-regressive next-token prediction with role-specific tokens, DoT
  facilitates seamless transitions between proposing ideas and critically
  evaluating them, providing richer feedback than binary signals. Furthermore, we
  formalize the DoT framework using Topos Theory, providing a mathematical
  foundation that ensures logical consistency and soundness in the reasoning
  process. This approach enhances both the training and inference processes
  within a single LLM, eliminating the need for multiple models or external
  control mechanisms. DoT offers a conceptual framework for designing
  next-generation reasoning-specialized models, emphasizing training efficiency,
  robust reasoning capabilities, and theoretical grounding. The code is available
  at https://github.com/diagram-of-thought/diagram-of-thought.
paper_authors._id: 66e928df2ff641e80cf92362
paper_authors.name: Yifan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-17T07:11:13.884Z
paper_authors._id: 647bf082aba7062fe5c51ca9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647bf082aba7062fe5c51ca9/p4lY9IjHiWZETKmFq1mtU.jpeg
paper_authors.fullname: Yifan Zhang
paper_authors.type: user
paper_authors.user: yifAI
paper_authors._id: 66e928df2ff641e80cf92363
paper_authors.name: Yang Yuan
paper_authors._id: 66e928df2ff641e80cf92364
paper_authors.name: Andrew Chi-Chih Yao
paper_title: On the Diagram of Thought
paper_summary: We introduce Diagram of Thought (DoT), a framework that models iterative
  reasoning in large language models (LLMs) as the construction of a directed
  acyclic graph (DAG) within a single model. Unlike traditional approaches that
  represent reasoning as linear chains or trees, DoT organizes propositions,
  critiques, refinements, and verifications into a cohesive DAG structure,
  allowing the model to explore complex reasoning pathways while maintaining
  logical consistency. Each node in the diagram corresponds to a proposition that
  has been proposed, critiqued, refined, or verified, enabling the LLM to
  iteratively improve its reasoning through natural language feedback. By
  leveraging auto-regressive next-token prediction with role-specific tokens, DoT
  facilitates seamless transitions between proposing ideas and critically
  evaluating them, providing richer feedback than binary signals. Furthermore, we
  formalize the DoT framework using Topos Theory, providing a mathematical
  foundation that ensures logical consistency and soundness in the reasoning
  process. This approach enhances both the training and inference processes
  within a single LLM, eliminating the need for multiple models or external
  control mechanisms. DoT offers a conceptual framework for designing
  next-generation reasoning-specialized models, emphasizing training efficiency,
  robust reasoning capabilities, and theoretical grounding. The code is available
  at https://github.com/diagram-of-thought/diagram-of-thought.
paper_ai_summary: The Diagram of Thought (DoT) framework models iterative reasoning in LLMs as a DAG, leveraging auto-regressive prediction and Topos Theory to enhance reasoning consistency and efficiency.
paper_ai_keywords: Diagram of Thought (DoT) large language models (LLMs) directed acyclic graph (DAG) iterative reasoning propositions critiques refinements verifications auto-regressive next-token prediction role-specific tokens Topos Theory logical consistency soundness