Think Before You Act: Decision Transformers with Internal Working Memory
Large language model (LLM)-based decision-making agents have shown the
ability to generalize across multiple tasks. However, their performance relies
on massive data and compute. We argue that this inefficiency stems from the
forgetting phenomenon, in which a model memorizes its behaviors in parameters
throughout training. As a result, training on a new task may deteriorate the
model's performance on previous tasks. In contrast to LLMs' implicit memory
mechanism, the human brain utilizes distributed memory storage, which helps
manage and organize multiple skills efficiently, mitigating the forgetting
phenomenon. Thus inspired, we propose an internal working memory module to
store, blend, and retrieve information for different downstream tasks.
Evaluation results show that the proposed method improves training efficiency
and generalization in both Atari games and meta-world object manipulation
tasks. Moreover, we demonstrate that memory fine-tuning further enhances the
adaptability of the proposed architecture.
647417dc7be71eb8b1ba3e34
Jikun Kang
claimed_verified
2023-06-30T20:56:05.834Z
6489e10ca13f65198dc6e122
/avatars/4aa9eab488157711b2f0298ddadee2f4.svg
Kang
user
JaxonK
647417dc7be71eb8b1ba3e35
Romain Laroche
647417dc7be71eb8b1ba3e36
Xindi Yuan
extracted_confirmed
2023-07-20T03:00:02.134Z
6495947b3eaaf416d924daeb
/avatars/2e79b75e0f5f54f11471affc2bb377b6.svg
Eric Yuan
user
eryua
647417dc7be71eb8b1ba3e37
Adam Trischler
647417dc7be71eb8b1ba3e38
Xue Liu
admin_assigned
2023-05-31T09:19:23.465Z
636e0d38a0d2db56c7ea3cb7
/avatars/670f122e26beb0041b6fa4c481cd2006.svg
liuxue
user
liuxue
647417dc7be71eb8b1ba3e39
Jie Fu
claimed_verified
2023-06-02T07:16:26.322Z
641a6895fb5ffff5ac79d593
https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/vxvwsto3llOEWGqQKGMYx.jpeg
Jie Fu
user
bigaidream
Think Before You Act: Decision Transformers with Internal Working Memory
Large language model (LLM)-based decision-making agents have shown the
ability to generalize across multiple tasks. However, their performance relies
on massive data and compute. We argue that this inefficiency stems from the
forgetting phenomenon, in which a model memorizes its behaviors in parameters
throughout training. As a result, training on a new task may deteriorate the
model's performance on previous tasks. In contrast to LLMs' implicit memory
mechanism, the human brain utilizes distributed memory storage, which helps
manage and organize multiple skills efficiently, mitigating the forgetting
phenomenon. Thus inspired, we propose an internal working memory module to
store, blend, and retrieve information for different downstream tasks.
Evaluation results show that the proposed method improves training efficiency
and generalization in both Atari games and meta-world object manipulation
tasks. Moreover, we demonstrate that memory fine-tuning further enhances the
adaptability of the proposed architecture.
An internal working memory module improves training efficiency and generalization for decision-making in Atari and meta-world tasks by addressing the forgetting phenomenon.
large language model decision-making agents forgetting phenomenon internal working memory module memory fine-tuning