title: Landscape of Thoughts: Visualizing the Reasoning Process of Large
    Language Models
summary: Numerous applications of large language models (LLMs) rely on their ability
  to perform step-by-step reasoning. However, the reasoning behavior of LLMs
  remains poorly understood, posing challenges to research, development, and
  safety. To address this gap, we introduce landscape of thoughts-the first
  visualization tool for users to inspect the reasoning paths of chain-of-thought
  and its derivatives on any multi-choice dataset. Specifically, we represent the
  states in a reasoning path as feature vectors that quantify their distances to
  all answer choices. These features are then visualized in two-dimensional plots
  using t-SNE. Qualitative and quantitative analysis with the landscape of
  thoughts effectively distinguishes between strong and weak models, correct and
  incorrect answers, as well as different reasoning tasks. It also uncovers
  undesirable reasoning patterns, such as low consistency and high uncertainty.
  Additionally, users can adapt our tool to a model that predicts the property
  they observe. We showcase this advantage by adapting our tool to a lightweight
  verifier that evaluates the correctness of reasoning paths. The code is
  publicly available at: https://github.com/tmlr-group/landscape-of-thoughts.
paper_authors._id: 67ecc00ea0e64cb1b27da958
paper_authors.name: Zhanke Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:21:26.334Z
paper_authors._id: 652b86ffa2d97e682b425b29
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/652b86ffa2d97e682b425b29/W_LzRdzndvKxqbw0ZP_BV.jpeg
paper_authors.fullname: (Andrew) Zhanke Zhou
paper_authors.type: user
paper_authors.user: AndrewZhou924
paper_authors._id: 67ecc00ea0e64cb1b27da959
paper_authors.name: Zhaocheng Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-03T08:28:45.477Z
paper_authors._id: 66554db35aa40da4de2194bc
paper_authors.avatarUrl: /avatars/fb130a8c2cc922eb125c6c60d457ce21.svg
paper_authors.fullname: Zhaocheng Zhu
paper_authors.type: user
paper_authors.user: KiddoZhu
paper_authors._id: 67ecc00ea0e64cb1b27da95a
paper_authors.name: Xuan Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:21:22.992Z
paper_authors._id: 649842ada9a46fd69e1d5121
paper_authors.avatarUrl: /avatars/35372b654678df75a2e8f4e8d20c5dcd.svg
paper_authors.fullname: XuanLi
paper_authors.type: user
paper_authors.user: GazeEzio
paper_authors._id: 67ecc00ea0e64cb1b27da95b
paper_authors.name: Mikhail Galkin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T19:55:33.633Z
paper_authors._id: 63c09599dd793d5a62890e7d
paper_authors.avatarUrl: /avatars/fed51ddd492b98e7cd4c3d1f82998635.svg
paper_authors.fullname: Michael Galkin
paper_authors.type: user
paper_authors.user: mgalkin
paper_authors._id: 67ecc00ea0e64cb1b27da95c
paper_authors.name: Xiao Feng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:21:20.033Z
paper_authors._id: 67bc2858b24be20e87b11c8e
paper_authors.avatarUrl: /avatars/a4cc2c8735fbcbeb3b182740749e521f.svg
paper_authors.fullname: Xiao Feng
paper_authors.type: user
paper_authors.user: KyrieXiao
paper_authors._id: 67ecc00ea0e64cb1b27da95d
paper_authors.name: Sanmi Koyejo
paper_authors._id: 67ecc00ea0e64cb1b27da95e
paper_authors.name: Jian Tang
paper_authors._id: 67ecc00ea0e64cb1b27da95f
paper_authors.name: Bo Han
paper_title: Landscape of Thoughts: Visualizing the Reasoning Process of Large
    Language Models
paper_summary: Numerous applications of large language models (LLMs) rely on their ability
  to perform step-by-step reasoning. However, the reasoning behavior of LLMs
  remains poorly understood, posing challenges to research, development, and
  safety. To address this gap, we introduce landscape of thoughts-the first
  visualization tool for users to inspect the reasoning paths of chain-of-thought
  and its derivatives on any multi-choice dataset. Specifically, we represent the
  states in a reasoning path as feature vectors that quantify their distances to
  all answer choices. These features are then visualized in two-dimensional plots
  using t-SNE. Qualitative and quantitative analysis with the landscape of
  thoughts effectively distinguishes between strong and weak models, correct and
  incorrect answers, as well as different reasoning tasks. It also uncovers
  undesirable reasoning patterns, such as low consistency and high uncertainty.
  Additionally, users can adapt our tool to a model that predicts the property
  they observe. We showcase this advantage by adapting our tool to a lightweight
  verifier that evaluates the correctness of reasoning paths. The code is
  publicly available at: https://github.com/tmlr-group/landscape-of-thoughts.
paper_ai_summary: A visualization tool named Landscape of Thoughts helps inspect and analyze reasoning paths of large language models on multi-choice datasets, identifying model strengths, correct answers, and reasoning inconsistencies.
paper_ai_keywords: large language models chain-of-thought feature vectors t-SNE qualitative analysis quantitative analysis reasoning paths model adaptation lightweight verifier