Self-Generated In-Context Examples Improve LLM Agents for Sequential
  Decision-Making Tasks
Many methods for improving Large Language Model (LLM) agents for sequential
decision-making tasks depend on task-specific knowledge engineering--such as
prompt tuning, curated in-context examples, or customized observation and
action spaces. Using these approaches, agent performance improves with the
quality or amount of knowledge engineering invested. Instead, we investigate
how LLM agents can automatically improve their performance by learning
in-context from their own successful experiences on similar tasks. Rather than
relying on task-specific knowledge engineering, we focus on constructing and
refining a database of self-generated examples. We demonstrate that even a
naive accumulation of successful trajectories across training tasks boosts test
performance on three benchmarks: ALFWorld (73% to 89%), Wordcraft (55% to 64%),
and InterCode-SQL (75% to 79%)--matching the performance the initial agent
achieves if allowed two to three attempts per task. We then introduce two
extensions: (1) database-level selection through population-based training to
identify high-performing example collections, and (2) exemplar-level selection
that retains individual trajectories based on their empirical utility as
in-context examples. These extensions further enhance performance, achieving
91% on ALFWorld--matching more complex approaches that employ task-specific
components and prompts. Our results demonstrate that automatic trajectory
database construction offers a compelling alternative to labor-intensive
knowledge engineering.
6814efeca3a0d78663350246
Vishnu Sarukkai
claimed_verified
2025-05-04T10:04:36.265Z
63051ebaacc17ce4ad347b24
/avatars/d6537b41f9275b90b5080713f273558c.svg
Vishnu Sarukkai
user
vsanimator
6814efeca3a0d78663350247
Zhiqiang Xie
admin_assigned
2025-05-05T07:45:38.125Z
6548b1d836332b0eede044cc
/avatars/c7946ff1783c342d4fc657ee3dc8a583.svg
Zhiqiang Xie
user
xiezhq
6814efeca3a0d78663350248
Kayvon Fatahalian
admin_assigned
2025-05-05T07:45:47.588Z
6501ceb6d6583ade3dff5399
/avatars/07fff5e56b9969642060acc43dd238fe.svg
Kayvon Fatahalian
user
kayvonf
Self-Generated In-Context Examples Improve LLM Agents for Sequential
  Decision-Making Tasks
Many methods for improving Large Language Model (LLM) agents for sequential
decision-making tasks depend on task-specific knowledge engineering--such as
prompt tuning, curated in-context examples, or customized observation and
action spaces. Using these approaches, agent performance improves with the
quality or amount of knowledge engineering invested. Instead, we investigate
how LLM agents can automatically improve their performance by learning
in-context from their own successful experiences on similar tasks. Rather than
relying on task-specific knowledge engineering, we focus on constructing and
refining a database of self-generated examples. We demonstrate that even a
naive accumulation of successful trajectories across training tasks boosts test
performance on three benchmarks: ALFWorld (73% to 89%), Wordcraft (55% to 64%),
and InterCode-SQL (75% to 79%)--matching the performance the initial agent
achieves if allowed two to three attempts per task. We then introduce two
extensions: (1) database-level selection through population-based training to
identify high-performing example collections, and (2) exemplar-level selection
that retains individual trajectories based on their empirical utility as
in-context examples. These extensions further enhance performance, achieving
91% on ALFWorld--matching more complex approaches that employ task-specific
components and prompts. Our results demonstrate that automatic trajectory
database construction offers a compelling alternative to labor-intensive
knowledge engineering.
LLM agents can improve performance on sequential decision-making tasks by learning from self-generated examples, enhancing test performance across multiple benchmarks without task-specific knowledge engineering.
Large Language Model prompt tuning in-context examples successful trajectories population-based training exemplar-level selection ALFWorld Wordcraft InterCode-SQL