Rethinking Reflection in Pre-Training
A language model's ability to reflect on its own reasoning provides a key
advantage for solving complex problems. While most recent research has focused
on how this ability develops during reinforcement learning, we show that it
actually begins to emerge much earlier - during the model's pre-training. To
study this, we introduce deliberate errors into chains-of-thought and test
whether the model can still arrive at the correct answer by recognizing and
correcting these mistakes. By tracking performance across different stages of
pre-training, we observe that this self-correcting ability appears early and
improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
trillion tokens displays self-correction on our six self-reflection tasks.
67f46e67a5ea8948d60c61af
Essential AI
claimed_verified
2025-04-08T06:52:15.045Z
67f40efb64c87965b9063655
/avatars/a25abd6f3318036c8d589633221c2010.svg
Research at Essential AI
user
Research-EAI
67f46e67a5ea8948d60c61b1
Darsh J Shah
claimed_verified
2025-04-08T06:52:17.485Z
66281933ac977c4c81bc0130
/avatars/83b3770be032ec4d20c03170198e4b0e.svg
Darsh Shah
user
darsh-essential
67f46e67a5ea8948d60c61b2
Peter Rushton
claimed_verified
2025-04-08T06:51:46.527Z
667f4ed15555b56581ad2b78
/avatars/18f0a166f5d4a34c4bd536655c478516.svg
Pete Rushton
user
prushto
67f46e67a5ea8948d60c61b3
Somanshu Singla
claimed_verified
2025-04-08T06:52:19.554Z
679bc0b23e12a166672e5275
/avatars/fe9d0e79c21c9d3594420e69e3809f0f.svg
Somanshu Singla
user
SomanshuSingla
67f46e67a5ea8948d60c61b4
Mohit Parmar
claimed_verified
2025-04-08T20:06:37.283Z
671aeee626d87414f59ee7d6
/avatars/131d0ff4c12bc70e74945910ce0e1d56.svg
Mohit Parmar
user
mohit-essential
67f46e67a5ea8948d60c61b5
Kurt Smith
claimed_verified
2025-04-09T07:37:49.297Z
678ff147044aeb2bd55a515f
/avatars/35356f9559b5ecaae33866b76411f54b.svg
kurt smith
user
kurt-essential
67f46e67a5ea8948d60c61b6
Yash Vanjani
claimed_verified
2025-04-08T06:51:49.116Z
66f5f3f99ef08fe3c1f4c35a
/avatars/41b8b6f90eb87e685b74587317296a1b.svg
Yash Vanjani
user
yash-essential
67f46e67a5ea8948d60c61b7
Ashish Vaswani
claimed_verified
2025-04-09T16:22:41.360Z
67f475769377f28d5d350412
/avatars/25413dc62dfb1e3ec18deeaec1458dce.svg
Ashish Vaswani
user
essentialite
67f46e67a5ea8948d60c61b8
Adarsh Chaluvaraju
claimed_verified
2025-05-29T20:33:36.418Z
67bfd6daca6e3c22b6de31ee
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/TTt6_o9tYEgeWK26DgI_7.png
Adarsh Chaluvaraju
user
cadarsh-essential
67f46e67a5ea8948d60c61b9
Andrew Hojel
67f46e67a5ea8948d60c61ba
Andrew Ma
67f46e67a5ea8948d60c61bb
Anil Thomas
67f46e67a5ea8948d60c61bc
Anthony Polloreno
admin_assigned
2025-04-09T16:55:05.160Z
6675e3ed66c4fa6d0c10e229
/avatars/f73c347d824a56079729c82d60d3edc3.svg
Anthony Polloreno
user
ampolloreno
67f46e67a5ea8948d60c61bd
Ashish Tanwer
claimed_verified
2025-04-09T07:37:52.090Z
63ea930ad0b894bbc7789f87
/avatars/14a6fda093779819b3c7a5487456b9ff.svg
Ashish Tanwer
user
ashishtanwer
67f46e67a5ea8948d60c61be
Burhan Drak Sibai
admin_assigned
2025-04-09T16:53:55.084Z
67ca1a86f2c1aacaf5e50a7f
/avatars/7db719864c315709a4bdcab088f902b7.svg
Burhan Drak Sibai
user
burhandraksibai
67f46e67a5ea8948d60c61bf
Divya S Mansingka
admin_assigned
2025-04-09T16:53:47.373Z
67f47db060c831a863066870
/avatars/fd502d8da5bf2f370ce623000f9349fa.svg
Divya Mansingka
user
dsm777
67f46e67a5ea8948d60c61c0
Divya Shivaprasad
admin_assigned
2025-04-09T16:53:36.948Z
67f4918916d6a4d15c805fc4
/avatars/1dca8ff7d5a74f7dcdc0085c477de1ec.svg
Divya Shivaprasad
user
divyashivaprasad
67f46e67a5ea8948d60c61c1
Ishaan Shah
admin_assigned
2025-04-09T16:53:28.302Z
65ef97d0e5fc4abe66c05ed0
/avatars/1d601a22639b3136bfb3519826451ddb.svg
Ishaan Shah
user
ishaan-essential
67f46e67a5ea8948d60c61c2
Karl Stratos
admin_assigned
2025-04-09T16:54:58.807Z
64d9ac38badf1110f7fcf030
/avatars/c55c61af8dd52e6b4856684638b850a6.svg
Karl Stratos
user
karlstratos
67f46e67a5ea8948d60c61c3
Khoi Nguyen
claimed_verified
2025-04-08T20:06:35.353Z
67ed7aa9290a7f9d33113fb5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jMMTObGZktuXWJ3wVVSxj.png
Khoi Nguyen
user
KTLK
67f46e67a5ea8948d60c61c4
Michael Callahan
claimed_verified
2025-06-23T08:33:17.079Z
66f62e1aa854f31fea6c7d25
/avatars/3cf6f471e26cec430fa5623bac938fa7.svg
Michael Callahan
user
mikecallahan
67f46e67a5ea8948d60c61c5
Michael Pust
admin_assigned
2025-04-09T16:54:27.866Z
66cd078ea796074d428fde0f
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qThw3H6ukqIAuRy7aJTN1.jpeg
Michael Pust
user
essentialpust
67f46e67a5ea8948d60c61c6
Mrinal Iyer
admin_assigned
2025-04-09T16:54:17.134Z
64c1826d561398def764229e
/avatars/18ce92d074a4900284e26b15bb48b14c.svg
Mrinal Iyer
user
mrinal-essential
67f46e67a5ea8948d60c61c7
Philip Monk
admin_assigned
2025-04-09T16:54:35.161Z
66622dacec18341b268f97a6
/avatars/8bc7d6a7c28c83aacdbeeb770716b1c0.svg
Philip Monk
user
monk-essential
67f46e67a5ea8948d60c61c8
Platon Mazarakis
admin_assigned
2025-04-09T16:54:41.949Z
67f4ced58c4cbc2f5d95cd17
/avatars/3f440a59c38f5c0c7a77746ef54ed0a5.svg
Platon Mazarakis
user
Platona
67f46e67a5ea8948d60c61c9
Ritvik Kapila
admin_assigned
2025-04-09T16:53:17.385Z
654bdcf2e06d25def57cc54b
/avatars/2d2612bd7072edd60876b504345fbf25.svg
Ritvik Kapila
user
rkapila
67f46e67a5ea8948d60c61ca
Saurabh Srivastava
claimed_verified
2025-04-08T06:51:51.930Z
66f59888850df13c40e41e03
/avatars/8667bed8cf0a8307f90ad448963dfae4.svg
Saurabh Srivastava
user
saurabh-essential
67f46e67a5ea8948d60c61cb
Tim Romanski
admin_assigned
2025-04-09T16:53:06.907Z
67101a7165442ddc48cb4b07
/avatars/551777fcd1638998ad9fd16804b313ec.svg
Tim Romanski
user
tim-essential
Rethinking Reflection in Pre-Training
A language model's ability to reflect on its own reasoning provides a key
advantage for solving complex problems. While most recent research has focused
on how this ability develops during reinforcement learning, we show that it
actually begins to emerge much earlier - during the model's pre-training. To
study this, we introduce deliberate errors into chains-of-thought and test
whether the model can still arrive at the correct answer by recognizing and
correcting these mistakes. By tracking performance across different stages of
pre-training, we observe that this self-correcting ability appears early and
improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
trillion tokens displays self-correction on our six self-reflection tasks.
Models exhibit self-correcting ability during pre-training by recognizing and addressing errors in their reasoning, a skill that improves over time.
language model self-reflection reinforcement learning pre-training chains-of-thought self-correcting ability OLMo2-7B tokens self-reflection tasks