title: Think Before You Act: Decision Transformers with Internal Working Memory
summary: Large language model (LLM)-based decision-making agents have shown the
  ability to generalize across multiple tasks. However, their performance relies
  on massive data and compute. We argue that this inefficiency stems from the
  forgetting phenomenon, in which a model memorizes its behaviors in parameters
  throughout training. As a result, training on a new task may deteriorate the
  model's performance on previous tasks. In contrast to LLMs' implicit memory
  mechanism, the human brain utilizes distributed memory storage, which helps
  manage and organize multiple skills efficiently, mitigating the forgetting
  phenomenon. Thus inspired, we propose an internal working memory module to
  store, blend, and retrieve information for different downstream tasks.
  Evaluation results show that the proposed method improves training efficiency
  and generalization in both Atari games and meta-world object manipulation
  tasks. Moreover, we demonstrate that memory fine-tuning further enhances the
  adaptability of the proposed architecture.
paper_authors._id: 647417dc7be71eb8b1ba3e34
paper_authors.name: Jikun Kang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-30T20:56:05.834Z
paper_authors._id: 6489e10ca13f65198dc6e122
paper_authors.avatarUrl: /avatars/4aa9eab488157711b2f0298ddadee2f4.svg
paper_authors.fullname: Kang
paper_authors.type: user
paper_authors.user: JaxonK
paper_authors._id: 647417dc7be71eb8b1ba3e35
paper_authors.name: Romain Laroche
paper_authors._id: 647417dc7be71eb8b1ba3e36
paper_authors.name: Xindi Yuan
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-07-20T03:00:02.134Z
paper_authors._id: 6495947b3eaaf416d924daeb
paper_authors.avatarUrl: /avatars/2e79b75e0f5f54f11471affc2bb377b6.svg
paper_authors.fullname: Eric Yuan
paper_authors.type: user
paper_authors.user: eryua
paper_authors._id: 647417dc7be71eb8b1ba3e37
paper_authors.name: Adam Trischler
paper_authors._id: 647417dc7be71eb8b1ba3e38
paper_authors.name: Xue Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-31T09:19:23.465Z
paper_authors._id: 636e0d38a0d2db56c7ea3cb7
paper_authors.avatarUrl: /avatars/670f122e26beb0041b6fa4c481cd2006.svg
paper_authors.fullname: liuxue
paper_authors.type: user
paper_authors.user: liuxue
paper_authors._id: 647417dc7be71eb8b1ba3e39
paper_authors.name: Jie Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-02T07:16:26.322Z
paper_authors._id: 641a6895fb5ffff5ac79d593
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/vxvwsto3llOEWGqQKGMYx.jpeg
paper_authors.fullname: Jie Fu
paper_authors.type: user
paper_authors.user: bigaidream
paper_title: Think Before You Act: Decision Transformers with Internal Working Memory
paper_summary: Large language model (LLM)-based decision-making agents have shown the
  ability to generalize across multiple tasks. However, their performance relies
  on massive data and compute. We argue that this inefficiency stems from the
  forgetting phenomenon, in which a model memorizes its behaviors in parameters
  throughout training. As a result, training on a new task may deteriorate the
  model's performance on previous tasks. In contrast to LLMs' implicit memory
  mechanism, the human brain utilizes distributed memory storage, which helps
  manage and organize multiple skills efficiently, mitigating the forgetting
  phenomenon. Thus inspired, we propose an internal working memory module to
  store, blend, and retrieve information for different downstream tasks.
  Evaluation results show that the proposed method improves training efficiency
  and generalization in both Atari games and meta-world object manipulation
  tasks. Moreover, we demonstrate that memory fine-tuning further enhances the
  adaptability of the proposed architecture.
paper_ai_summary: An internal working memory module improves training efficiency and generalization for decision-making in Atari and meta-world tasks by addressing the forgetting phenomenon.
paper_ai_keywords: large language model decision-making agents forgetting phenomenon internal working memory module memory fine-tuning