title: Rethinking Reflection in Pre-Training
summary: A language model's ability to reflect on its own reasoning provides a key
  advantage for solving complex problems. While most recent research has focused
  on how this ability develops during reinforcement learning, we show that it
  actually begins to emerge much earlier - during the model's pre-training. To
  study this, we introduce deliberate errors into chains-of-thought and test
  whether the model can still arrive at the correct answer by recognizing and
  correcting these mistakes. By tracking performance across different stages of
  pre-training, we observe that this self-correcting ability appears early and
  improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
  trillion tokens displays self-correction on our six self-reflection tasks.
paper_authors._id: 67f46e67a5ea8948d60c61af
paper_authors.name: Essential AI
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:52:15.045Z
paper_authors._id: 67f40efb64c87965b9063655
paper_authors.avatarUrl: /avatars/a25abd6f3318036c8d589633221c2010.svg
paper_authors.fullname: Research at Essential AI
paper_authors.type: user
paper_authors.user: Research-EAI
paper_authors._id: 67f46e67a5ea8948d60c61b1
paper_authors.name: Darsh J Shah
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:52:17.485Z
paper_authors._id: 66281933ac977c4c81bc0130
paper_authors.avatarUrl: /avatars/83b3770be032ec4d20c03170198e4b0e.svg
paper_authors.fullname: Darsh Shah
paper_authors.type: user
paper_authors.user: darsh-essential
paper_authors._id: 67f46e67a5ea8948d60c61b2
paper_authors.name: Peter Rushton
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:51:46.527Z
paper_authors._id: 667f4ed15555b56581ad2b78
paper_authors.avatarUrl: /avatars/18f0a166f5d4a34c4bd536655c478516.svg
paper_authors.fullname: Pete Rushton
paper_authors.type: user
paper_authors.user: prushto
paper_authors._id: 67f46e67a5ea8948d60c61b3
paper_authors.name: Somanshu Singla
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:52:19.554Z
paper_authors._id: 679bc0b23e12a166672e5275
paper_authors.avatarUrl: /avatars/fe9d0e79c21c9d3594420e69e3809f0f.svg
paper_authors.fullname: Somanshu Singla
paper_authors.type: user
paper_authors.user: somanshu-essential
paper_authors._id: 67f46e67a5ea8948d60c61b4
paper_authors.name: Mohit Parmar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T20:06:37.283Z
paper_authors._id: 671aeee626d87414f59ee7d6
paper_authors.avatarUrl: /avatars/131d0ff4c12bc70e74945910ce0e1d56.svg
paper_authors.fullname: Mohit Parmar
paper_authors.type: user
paper_authors.user: mohit-essential
paper_authors._id: 67f46e67a5ea8948d60c61b5
paper_authors.name: Kurt Smith
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-09T07:37:49.297Z
paper_authors._id: 678ff147044aeb2bd55a515f
paper_authors.avatarUrl: /avatars/35356f9559b5ecaae33866b76411f54b.svg
paper_authors.fullname: kurt smith
paper_authors.type: user
paper_authors.user: kurt-essential
paper_authors._id: 67f46e67a5ea8948d60c61b6
paper_authors.name: Yash Vanjani
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:51:49.116Z
paper_authors._id: 66f5f3f99ef08fe3c1f4c35a
paper_authors.avatarUrl: /avatars/41b8b6f90eb87e685b74587317296a1b.svg
paper_authors.fullname: Yash Vanjani
paper_authors.type: user
paper_authors.user: yash-essential
paper_authors._id: 67f46e67a5ea8948d60c61b7
paper_authors.name: Ashish Vaswani
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-09T16:22:41.360Z
paper_authors._id: 67f475769377f28d5d350412
paper_authors.avatarUrl: /avatars/25413dc62dfb1e3ec18deeaec1458dce.svg
paper_authors.fullname: Ashish Vaswani
paper_authors.type: user
paper_authors.user: essentialite
paper_authors._id: 67f46e67a5ea8948d60c61b8
paper_authors.name: Adarsh Chaluvaraju
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T20:33:36.418Z
paper_authors._id: 67bfd6daca6e3c22b6de31ee
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/TTt6_o9tYEgeWK26DgI_7.png
paper_authors.fullname: Adarsh Chaluvaraju
paper_authors.type: user
paper_authors.user: cadarsh-essential
paper_authors._id: 67f46e67a5ea8948d60c61b9
paper_authors.name: Andrew Hojel
paper_authors._id: 67f46e67a5ea8948d60c61ba
paper_authors.name: Andrew Ma
paper_authors._id: 67f46e67a5ea8948d60c61bb
paper_authors.name: Anil Thomas
paper_authors._id: 67f46e67a5ea8948d60c61bc
paper_authors.name: Anthony Polloreno
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:55:05.160Z
paper_authors._id: 6675e3ed66c4fa6d0c10e229
paper_authors.avatarUrl: /avatars/f73c347d824a56079729c82d60d3edc3.svg
paper_authors.fullname: Anthony Polloreno
paper_authors.type: user
paper_authors.user: ampolloreno
paper_authors._id: 67f46e67a5ea8948d60c61bd
paper_authors.name: Ashish Tanwer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-09T07:37:52.090Z
paper_authors._id: 63ea930ad0b894bbc7789f87
paper_authors.avatarUrl: /avatars/14a6fda093779819b3c7a5487456b9ff.svg
paper_authors.fullname: Ashish Tanwer
paper_authors.type: user
paper_authors.user: ashishtanwer
paper_authors._id: 67f46e67a5ea8948d60c61be
paper_authors.name: Burhan Drak Sibai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:55.084Z
paper_authors._id: 67ca1a86f2c1aacaf5e50a7f
paper_authors.avatarUrl: /avatars/7db719864c315709a4bdcab088f902b7.svg
paper_authors.fullname: Burhan Drak Sibai
paper_authors.type: user
paper_authors.user: burhandraksibai
paper_authors._id: 67f46e67a5ea8948d60c61bf
paper_authors.name: Divya S Mansingka
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:47.373Z
paper_authors._id: 67f47db060c831a863066870
paper_authors.avatarUrl: /avatars/fd502d8da5bf2f370ce623000f9349fa.svg
paper_authors.fullname: Divya Mansingka
paper_authors.type: user
paper_authors.user: dsm777
paper_authors._id: 67f46e67a5ea8948d60c61c0
paper_authors.name: Divya Shivaprasad
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:36.948Z
paper_authors._id: 67f4918916d6a4d15c805fc4
paper_authors.avatarUrl: /avatars/1dca8ff7d5a74f7dcdc0085c477de1ec.svg
paper_authors.fullname: Divya Shivaprasad
paper_authors.type: user
paper_authors.user: divyashivaprasad
paper_authors._id: 67f46e67a5ea8948d60c61c1
paper_authors.name: Ishaan Shah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:28.302Z
paper_authors._id: 65ef97d0e5fc4abe66c05ed0
paper_authors.avatarUrl: /avatars/1d601a22639b3136bfb3519826451ddb.svg
paper_authors.fullname: Ishaan Shah
paper_authors.type: user
paper_authors.user: ishaan-essential
paper_authors._id: 67f46e67a5ea8948d60c61c2
paper_authors.name: Karl Stratos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:54:58.807Z
paper_authors._id: 64d9ac38badf1110f7fcf030
paper_authors.avatarUrl: /avatars/c55c61af8dd52e6b4856684638b850a6.svg
paper_authors.fullname: Karl Stratos
paper_authors.type: user
paper_authors.user: karlstratos
paper_authors._id: 67f46e67a5ea8948d60c61c3
paper_authors.name: Khoi Nguyen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T20:06:35.353Z
paper_authors._id: 67ed7aa9290a7f9d33113fb5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jMMTObGZktuXWJ3wVVSxj.png
paper_authors.fullname: Khoi Nguyen
paper_authors.type: user
paper_authors.user: KTLK
paper_authors._id: 67f46e67a5ea8948d60c61c4
paper_authors.name: Michael Callahan
paper_authors._id: 67f46e67a5ea8948d60c61c5
paper_authors.name: Michael Pust
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:54:27.866Z
paper_authors._id: 66cd078ea796074d428fde0f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qThw3H6ukqIAuRy7aJTN1.jpeg
paper_authors.fullname: Michael Pust
paper_authors.type: user
paper_authors.user: essentialpust
paper_authors._id: 67f46e67a5ea8948d60c61c6
paper_authors.name: Mrinal Iyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:54:17.134Z
paper_authors._id: 64c1826d561398def764229e
paper_authors.avatarUrl: /avatars/18ce92d074a4900284e26b15bb48b14c.svg
paper_authors.fullname: Mrinal Iyer
paper_authors.type: user
paper_authors.user: mrinal-essential
paper_authors._id: 67f46e67a5ea8948d60c61c7
paper_authors.name: Philip Monk
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:54:35.161Z
paper_authors._id: 66622dacec18341b268f97a6
paper_authors.avatarUrl: /avatars/8bc7d6a7c28c83aacdbeeb770716b1c0.svg
paper_authors.fullname: Philip Monk
paper_authors.type: user
paper_authors.user: monk-essential
paper_authors._id: 67f46e67a5ea8948d60c61c8
paper_authors.name: Platon Mazarakis
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:54:41.949Z
paper_authors._id: 67f4ced58c4cbc2f5d95cd17
paper_authors.avatarUrl: /avatars/3f440a59c38f5c0c7a77746ef54ed0a5.svg
paper_authors.fullname: Platon Mazarakis
paper_authors.type: user
paper_authors.user: Platona
paper_authors._id: 67f46e67a5ea8948d60c61c9
paper_authors.name: Ritvik Kapila
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:17.385Z
paper_authors._id: 654bdcf2e06d25def57cc54b
paper_authors.avatarUrl: /avatars/2d2612bd7072edd60876b504345fbf25.svg
paper_authors.fullname: Ritvik Kapila
paper_authors.type: user
paper_authors.user: rkapila
paper_authors._id: 67f46e67a5ea8948d60c61ca
paper_authors.name: Saurabh Srivastava
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:51:51.930Z
paper_authors._id: 66f59888850df13c40e41e03
paper_authors.avatarUrl: /avatars/8667bed8cf0a8307f90ad448963dfae4.svg
paper_authors.fullname: Saurabh Srivastava
paper_authors.type: user
paper_authors.user: saurabh-essential
paper_authors._id: 67f46e67a5ea8948d60c61cb
paper_authors.name: Tim Romanski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-09T16:53:06.907Z
paper_authors._id: 67101a7165442ddc48cb4b07
paper_authors.avatarUrl: /avatars/551777fcd1638998ad9fd16804b313ec.svg
paper_authors.fullname: Tim Romanski
paper_authors.type: user
paper_authors.user: tim-essential
paper_title: Rethinking Reflection in Pre-Training
paper_summary: A language model's ability to reflect on its own reasoning provides a key
  advantage for solving complex problems. While most recent research has focused
  on how this ability develops during reinforcement learning, we show that it
  actually begins to emerge much earlier - during the model's pre-training. To
  study this, we introduce deliberate errors into chains-of-thought and test
  whether the model can still arrive at the correct answer by recognizing and
  correcting these mistakes. By tracking performance across different stages of
  pre-training, we observe that this self-correcting ability appears early and
  improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
  trillion tokens displays self-correction on our six self-reflection tasks.
paper_ai_summary: Models exhibit self-correcting ability during pre-training by recognizing and addressing errors in their reasoning, a skill that improves over time.
paper_ai_keywords: language model self-reflection reinforcement learning pre-training chains-of-thought self-correcting ability OLMo2-7B tokens self-reflection tasks