title: Measuring Faithfulness in Chain-of-Thought Reasoning
summary: Large language models (LLMs) perform better when they produce step-by-step,
  "Chain-of-Thought" (CoT) reasoning before answering a question, but it is
  unclear if the stated reasoning is a faithful explanation of the model's actual
  reasoning (i.e., its process for answering the question). We investigate
  hypotheses for how CoT reasoning may be unfaithful, by examining how the model
  predictions change when we intervene on the CoT (e.g., by adding mistakes or
  paraphrasing it). Models show large variation across tasks in how strongly they
  condition on the CoT when predicting their answer, sometimes relying heavily on
  the CoT and other times primarily ignoring it. CoT's performance boost does not
  seem to come from CoT's added test-time compute alone or from information
  encoded via the particular phrasing of the CoT. As models become larger and
  more capable, they produce less faithful reasoning on most tasks we study.
  Overall, our results suggest that CoT can be faithful if the circumstances such
  as the model size and task are carefully chosen.
paper_authors._id: 64c1c4697e6975b3916e6ddd
paper_authors.name: Tamera Lanham
paper_authors._id: 64c1c4697e6975b3916e6dde
paper_authors.name: Anna Chen
paper_authors._id: 64c1c4697e6975b3916e6ddf
paper_authors.name: Ansh Radhakrishnan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:57:42.888Z
paper_authors._id: 625e0426e1176e6c772aa37c
paper_authors.avatarUrl: /avatars/a05c5e3b964ca2a6b82eec0e3d89c064.svg
paper_authors.fullname: Ansh Radhakrishnan
paper_authors.type: user
paper_authors.user: anshr
paper_authors._id: 64c1c4697e6975b3916e6de0
paper_authors.name: Benoit Steiner
paper_authors._id: 64c1c4697e6975b3916e6de1
paper_authors.name: Carson Denison
paper_authors._id: 64c1c4697e6975b3916e6de2
paper_authors.name: Danny Hernandez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:58:16.063Z
paper_authors._id: 63fe9f9452441fe3e878b631
paper_authors.avatarUrl: /avatars/de5cf5dc8b7e9d8d0dfef24850a4d4b3.svg
paper_authors.fullname: Danny Hernandez
paper_authors.type: user
paper_authors.user: DannyDev06
paper_authors._id: 64c1c4697e6975b3916e6de3
paper_authors.name: Dustin Li
paper_authors._id: 64c1c4697e6975b3916e6de4
paper_authors.name: Esin Durmus
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:58:53.907Z
paper_authors._id: 627dd7dbba19b19eff6aaa46
paper_authors.avatarUrl: /avatars/6e43025ccb96e48339d1db477da043b0.svg
paper_authors.fullname: Esin Durmus
paper_authors.type: user
paper_authors.user: esdurmus
paper_authors._id: 64c1c4697e6975b3916e6de5
paper_authors.name: Evan Hubinger
paper_authors._id: 64c1c4697e6975b3916e6de6
paper_authors.name: Jackson Kernion
paper_authors._id: 64c1c4697e6975b3916e6de7
paper_authors.name: Kamilė Lukošiūtė
paper_authors._id: 64c1c4697e6975b3916e6de8
paper_authors.name: Karina Nguyen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:59:15.889Z
paper_authors._id: 62ef4469cbe9a53fcd5a7f03
paper_authors.avatarUrl: /avatars/13352152a67ef304ba444cbc310be5dd.svg
paper_authors.fullname: Karina Nguyen
paper_authors.type: user
paper_authors.user: karinanguyen
paper_authors._id: 64c1c4697e6975b3916e6de9
paper_authors.name: Newton Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:59:24.513Z
paper_authors._id: 6230f73441ca6e8d8f614008
paper_authors.avatarUrl: /avatars/2c6f84c6ecb71c047c5d043d5a89f791.svg
paper_authors.fullname: Newton Cheng
paper_authors.type: user
paper_authors.user: nccheng
paper_authors._id: 64c1c4697e6975b3916e6dea
paper_authors.name: Nicholas Joseph
paper_authors._id: 64c1c4697e6975b3916e6deb
paper_authors.name: Nicholas Schiefer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T08:59:54.082Z
paper_authors._id: 63a246e2931d2c13ad842a59
paper_authors.avatarUrl: /avatars/9ac4f3d01c293ce97507be402dfa688c.svg
paper_authors.fullname: Nicholas Schiefer
paper_authors.type: user
paper_authors.user: nschiefer
paper_authors._id: 64c1c4697e6975b3916e6dec
paper_authors.name: Oliver Rausch
paper_authors._id: 64c1c4697e6975b3916e6ded
paper_authors.name: Robin Larson
paper_authors._id: 64c1c4697e6975b3916e6dee
paper_authors.name: Sam McCandlish
paper_authors._id: 64c1c4697e6975b3916e6def
paper_authors.name: Sandipan Kundu
paper_authors._id: 64c1c4697e6975b3916e6df0
paper_authors.name: Saurav Kadavath
paper_authors._id: 64c1c4697e6975b3916e6df1
paper_authors.name: Shannon Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T09:00:45.948Z
paper_authors._id: 62ce2249342b1d5dab9451f4
paper_authors.avatarUrl: /avatars/9a5f6992b7842ecf721c6908245d75ba.svg
paper_authors.fullname: Shannon Yang
paper_authors.type: user
paper_authors.user: shannonyang
paper_authors._id: 64c1c4697e6975b3916e6df2
paper_authors.name: Thomas Henighan
paper_authors._id: 64c1c4697e6975b3916e6df3
paper_authors.name: Timothy Maxwell
paper_authors._id: 64c1c4697e6975b3916e6df4
paper_authors.name: Timothy Telleen-Lawton
paper_authors._id: 64c1c4697e6975b3916e6df5
paper_authors.name: Tristan Hume
paper_authors._id: 64c1c4697e6975b3916e6df6
paper_authors.name: Zac Hatfield-Dodds
paper_authors._id: 64c1c4697e6975b3916e6df7
paper_authors.name: Jared Kaplan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-27T10:39:11.814Z
paper_authors._id: 633b051c7af633cbcd01b0ff
paper_authors.avatarUrl: /avatars/e2f315a8d6241ae1a783ba2957a211a5.svg
paper_authors.fullname: Jared Kaplan
paper_authors.type: user
paper_authors.user: FrizzleFried
paper_authors._id: 64c1c4697e6975b3916e6df8
paper_authors.name: Jan Brauner
paper_authors._id: 64c1c4697e6975b3916e6df9
paper_authors.name: Samuel R. Bowman
paper_authors._id: 64c1c4697e6975b3916e6dfa
paper_authors.name: Ethan Perez
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-07-27T01:12:09.591Z
paper_authors._id: 62bb28dc4d3faf9c8938f742
paper_authors.avatarUrl: /avatars/cceeb6b8e679b7460519024cd5e301b5.svg
paper_authors.fullname: Ethan
paper_authors.type: user
paper_authors.user: ethanjperez
paper_title: Measuring Faithfulness in Chain-of-Thought Reasoning
paper_summary: Large language models (LLMs) perform better when they produce step-by-step,
  "Chain-of-Thought" (CoT) reasoning before answering a question, but it is
  unclear if the stated reasoning is a faithful explanation of the model's actual
  reasoning (i.e., its process for answering the question). We investigate
  hypotheses for how CoT reasoning may be unfaithful, by examining how the model
  predictions change when we intervene on the CoT (e.g., by adding mistakes or
  paraphrasing it). Models show large variation across tasks in how strongly they
  condition on the CoT when predicting their answer, sometimes relying heavily on
  the CoT and other times primarily ignoring it. CoT's performance boost does not
  seem to come from CoT's added test-time compute alone or from information
  encoded via the particular phrasing of the CoT. As models become larger and
  more capable, they produce less faithful reasoning on most tasks we study.
  Overall, our results suggest that CoT can be faithful if the circumstances such
  as the model size and task are carefully chosen.
paper_ai_keywords: Chain-of-Thought (CoT) reasoning model predictions model conditioning intervention on the CoT test-time compute information encoded via the particular phrasing of the CoT model size and task