title: RL + Transformer = A General-Purpose Problem Solver
summary: What if artificial intelligence could not only solve problems for which it
  was trained but also learn to teach itself to solve new problems (i.e.,
  meta-learn)? In this study, we demonstrate that a pre-trained transformer
  fine-tuned with reinforcement learning over multiple episodes develops the
  ability to solve problems that it has never encountered before - an emergent
  ability called In-Context Reinforcement Learning (ICRL). This powerful
  meta-learner not only excels in solving unseen in-distribution environments
  with remarkable sample efficiency, but also shows strong performance in
  out-of-distribution environments. In addition, we show that it exhibits
  robustness to the quality of its training data, seamlessly stitches together
  behaviors from its context, and adapts to non-stationary environments. These
  behaviors demonstrate that an RL-trained transformer can iteratively improve
  upon its own solutions, making it an excellent general-purpose problem solver.
paper_authors._id: 67979f107dbf69e4e34cc51a
paper_authors.name: Micah Rentschler
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-06T17:21:33.919Z
paper_authors._id: 66fffe1b3ec4cc293d40f2d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66fffe1b3ec4cc293d40f2d5/-6Ff7rSQ_fvqBmTQRTlB4.png
paper_authors.fullname: Micah Rentschler
paper_authors.type: user
paper_authors.user: micahr234
paper_authors._id: 67979f107dbf69e4e34cc51b
paper_authors.name: Jesse Roberts
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:09:51.776Z
paper_authors._id: 63c19eb3a0ffa3857eae2efa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63c19eb3a0ffa3857eae2efa/pB-CbPa1QFTImiHanNCxU.jpeg
paper_authors.fullname: Jesse Roberts
paper_authors.type: user
paper_authors.user: JesseTNRoberts
paper_title: RL + Transformer = A General-Purpose Problem Solver
paper_summary: What if artificial intelligence could not only solve problems for which it
  was trained but also learn to teach itself to solve new problems (i.e.,
  meta-learn)? In this study, we demonstrate that a pre-trained transformer
  fine-tuned with reinforcement learning over multiple episodes develops the
  ability to solve problems that it has never encountered before - an emergent
  ability called In-Context Reinforcement Learning (ICRL). This powerful
  meta-learner not only excels in solving unseen in-distribution environments
  with remarkable sample efficiency, but also shows strong performance in
  out-of-distribution environments. In addition, we show that it exhibits
  robustness to the quality of its training data, seamlessly stitches together
  behaviors from its context, and adapts to non-stationary environments. These
  behaviors demonstrate that an RL-trained transformer can iteratively improve
  upon its own solutions, making it an excellent general-purpose problem solver.
paper_ai_summary: A pre-trained transformer fine-tuned with reinforcement learning develops the ability to solve unseen problems with sample efficiency and adaptability, demonstrating robust meta-learning.
paper_ai_keywords: transformer reinforcement learning In-Context Reinforcement Learning (ICRL) meta-learner sample efficiency out-of-distribution robustness behavior stitching non-stationary environments