title: Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
summary: We explore a method for improving the performance of large language models
  through self-reflection and reinforcement learning. By incentivizing the model
  to generate better self-reflections when it answers incorrectly, we demonstrate
  that a model's ability to solve complex, verifiable tasks can be enhanced even
  when generating synthetic data is infeasible and only binary feedback is
  available. Our framework operates in two stages: first, upon failing a given
  task, the model generates a self-reflective commentary analyzing its previous
  attempt; second, the model is given another attempt at the task with the
  self-reflection in context. If the subsequent attempt succeeds, the tokens
  generated during the self-reflection phase are rewarded. Our experimental
  results show substantial performance gains across a variety of model
  architectures, as high as 34.7% improvement at math equation writing and 18.1%
  improvement at function calling. Notably, smaller fine-tuned models (1.5
  billion to 7 billion parameters) outperform models in the same family that are
  10 times larger. Our novel paradigm is thus an exciting pathway to more useful
  and reliable language models that can self-improve on challenging tasks with
  limited external feedback.
paper_authors._id: 683ffca568402c738a947f4e
paper_authors.name: Shelly Bensal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:40:16.151Z
paper_authors._id: 6747c2ef833196ed191c33e4
paper_authors.avatarUrl: /avatars/3f2780c6470162e6b39c64c30440d862.svg
paper_authors.fullname: Shelly Bensal
paper_authors.type: user
paper_authors.user: shelly-writer
paper_authors._id: 683ffca568402c738a947f4f
paper_authors.name: Umar Jamil
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:40:22.194Z
paper_authors._id: 63805c4a9f1f158b014d63d8
paper_authors.avatarUrl: /avatars/3221a1ed4d3f809993ba5c79c341f88d.svg
paper_authors.fullname: Umar Jamil
paper_authors.type: user
paper_authors.user: hkproj
paper_authors._id: 683ffca568402c738a947f50
paper_authors.name: Christopher Bryant
paper_authors._id: 683ffca568402c738a947f51
paper_authors.name: Melisa Russak
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:53:45.047Z
paper_authors._id: 60e61b3969bd0df25c9375da
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1625692968400-noauth.jpeg
paper_authors.fullname: Melisa Russak
paper_authors.type: user
paper_authors.user: melisa
paper_authors._id: 683ffca568402c738a947f52
paper_authors.name: Kiran Kamble
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T15:03:25.497Z
paper_authors._id: 621d6f532165dc431641e438
paper_authors.avatarUrl: /avatars/56ccef10a8426d7160ef3586a771bd63.svg
paper_authors.fullname: Kiran Kamble
paper_authors.type: user
paper_authors.user: kiranr
paper_authors._id: 683ffca568402c738a947f53
paper_authors.name: Dmytro Mozolevskyi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:40:32.265Z
paper_authors._id: 64f13a7c9be8cab82d9b5a55
paper_authors.avatarUrl: /avatars/a00b5d386016697b4c4cc746bac16168.svg
paper_authors.fullname: Dmytro Mozolevskyi
paper_authors.type: user
paper_authors.user: dmytro-writer
paper_authors._id: 683ffca568402c738a947f54
paper_authors.name: Muayad Ali
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:40:38.833Z
paper_authors._id: 5f0647f35d08220171a0ae39
paper_authors.avatarUrl: /avatars/53cce20f4bc6204c692f021f4b40e0ff.svg
paper_authors.fullname: Muayad Ali
paper_authors.type: user
paper_authors.user: Goldenzero
paper_authors._id: 683ffca568402c738a947f55
paper_authors.name: Waseem AlShikh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T15:03:22.388Z
paper_authors._id: 60cd486d723acf5eb46fe8d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60cd486d723acf5eb46fe8d3/Z1bD1kjvZ0QAOjZna41Xr.jpeg
paper_authors.fullname: Waseem AlShikh
paper_authors.type: user
paper_authors.user: wassemgtk
paper_title: Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
paper_summary: We explore a method for improving the performance of large language models
  through self-reflection and reinforcement learning. By incentivizing the model
  to generate better self-reflections when it answers incorrectly, we demonstrate
  that a model's ability to solve complex, verifiable tasks can be enhanced even
  when generating synthetic data is infeasible and only binary feedback is
  available. Our framework operates in two stages: first, upon failing a given
  task, the model generates a self-reflective commentary analyzing its previous
  attempt; second, the model is given another attempt at the task with the
  self-reflection in context. If the subsequent attempt succeeds, the tokens
  generated during the self-reflection phase are rewarded. Our experimental
  results show substantial performance gains across a variety of model
  architectures, as high as 34.7% improvement at math equation writing and 18.1%
  improvement at function calling. Notably, smaller fine-tuned models (1.5
  billion to 7 billion parameters) outperform models in the same family that are
  10 times larger. Our novel paradigm is thus an exciting pathway to more useful
  and reliable language models that can self-improve on challenging tasks with
  limited external feedback.
paper_ai_summary: A method using self-reflection and reinforcement learning improves the performance of large language models, especially with limited feedback, by rewarding self-reflections that lead to better task performance.
paper_ai_keywords: self-reflection reinforcement learning self-reflective commentary performance gains math equation writing function calling parameter-efficient fine-tuning