title: Self-Generated In-Context Examples Improve LLM Agents for Sequential
    Decision-Making Tasks
summary: Many methods for improving Large Language Model (LLM) agents for sequential
  decision-making tasks depend on task-specific knowledge engineering--such as
  prompt tuning, curated in-context examples, or customized observation and
  action spaces. Using these approaches, agent performance improves with the
  quality or amount of knowledge engineering invested. Instead, we investigate
  how LLM agents can automatically improve their performance by learning
  in-context from their own successful experiences on similar tasks. Rather than
  relying on task-specific knowledge engineering, we focus on constructing and
  refining a database of self-generated examples. We demonstrate that even a
  naive accumulation of successful trajectories across training tasks boosts test
  performance on three benchmarks: ALFWorld (73% to 89%), Wordcraft (55% to 64%),
  and InterCode-SQL (75% to 79%)--matching the performance the initial agent
  achieves if allowed two to three attempts per task. We then introduce two
  extensions: (1) database-level selection through population-based training to
  identify high-performing example collections, and (2) exemplar-level selection
  that retains individual trajectories based on their empirical utility as
  in-context examples. These extensions further enhance performance, achieving
  91% on ALFWorld--matching more complex approaches that employ task-specific
  components and prompts. Our results demonstrate that automatic trajectory
  database construction offers a compelling alternative to labor-intensive
  knowledge engineering.
paper_authors._id: 6814efeca3a0d78663350246
paper_authors.name: Vishnu Sarukkai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-04T10:04:36.265Z
paper_authors._id: 63051ebaacc17ce4ad347b24
paper_authors.avatarUrl: /avatars/d6537b41f9275b90b5080713f273558c.svg
paper_authors.fullname: Vishnu Sarukkai
paper_authors.type: user
paper_authors.user: vsanimator
paper_authors._id: 6814efeca3a0d78663350247
paper_authors.name: Zhiqiang Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:45:38.125Z
paper_authors._id: 6548b1d836332b0eede044cc
paper_authors.avatarUrl: /avatars/c7946ff1783c342d4fc657ee3dc8a583.svg
paper_authors.fullname: Zhiqiang Xie
paper_authors.type: user
paper_authors.user: xiezhq
paper_authors._id: 6814efeca3a0d78663350248
paper_authors.name: Kayvon Fatahalian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T07:45:47.588Z
paper_authors._id: 6501ceb6d6583ade3dff5399
paper_authors.avatarUrl: /avatars/07fff5e56b9969642060acc43dd238fe.svg
paper_authors.fullname: Kayvon Fatahalian
paper_authors.type: user
paper_authors.user: kayvonf
paper_title: Self-Generated In-Context Examples Improve LLM Agents for Sequential
    Decision-Making Tasks
paper_summary: Many methods for improving Large Language Model (LLM) agents for sequential
  decision-making tasks depend on task-specific knowledge engineering--such as
  prompt tuning, curated in-context examples, or customized observation and
  action spaces. Using these approaches, agent performance improves with the
  quality or amount of knowledge engineering invested. Instead, we investigate
  how LLM agents can automatically improve their performance by learning
  in-context from their own successful experiences on similar tasks. Rather than
  relying on task-specific knowledge engineering, we focus on constructing and
  refining a database of self-generated examples. We demonstrate that even a
  naive accumulation of successful trajectories across training tasks boosts test
  performance on three benchmarks: ALFWorld (73% to 89%), Wordcraft (55% to 64%),
  and InterCode-SQL (75% to 79%)--matching the performance the initial agent
  achieves if allowed two to three attempts per task. We then introduce two
  extensions: (1) database-level selection through population-based training to
  identify high-performing example collections, and (2) exemplar-level selection
  that retains individual trajectories based on their empirical utility as
  in-context examples. These extensions further enhance performance, achieving
  91% on ALFWorld--matching more complex approaches that employ task-specific
  components and prompts. Our results demonstrate that automatic trajectory
  database construction offers a compelling alternative to labor-intensive
  knowledge engineering.
paper_ai_summary: LLM agents can improve performance on sequential decision-making tasks by learning from self-generated examples, enhancing test performance across multiple benchmarks without task-specific knowledge engineering.
paper_ai_keywords: Large Language Model prompt tuning in-context examples successful trajectories population-based training exemplar-level selection ALFWorld Wordcraft InterCode-SQL