title: Unraveling the Complexity of Memory in RL Agents: an Approach for
    Classification and Evaluation
summary: The incorporation of memory into agents is essential for numerous tasks
  within the domain of Reinforcement Learning (RL). In particular, memory is
  paramount for tasks that require the utilization of past information,
  adaptation to novel environments, and improved sample efficiency. However, the
  term ``memory'' encompasses a wide range of concepts, which, coupled with the
  lack of a unified methodology for validating an agent's memory, leads to
  erroneous judgments about agents' memory capabilities and prevents objective
  comparison with other memory-enhanced agents. This paper aims to streamline the
  concept of memory in RL by providing practical precise definitions of agent
  memory types, such as long-term versus short-term memory and declarative versus
  procedural memory, inspired by cognitive science. Using these definitions, we
  categorize different classes of agent memory, propose a robust experimental
  methodology for evaluating the memory capabilities of RL agents, and
  standardize evaluations. Furthermore, we empirically demonstrate the importance
  of adhering to the proposed methodology when evaluating different types of
  agent memory by conducting experiments with different RL agents and what its
  violation leads to.
paper_authors._id: 67583ace95bb6a9c895df272
paper_authors.name: Egor Cherepanov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-10T13:06:34.114Z
paper_authors._id: 6668687caee0993c95b0eb81
paper_authors.avatarUrl: /avatars/301fe1f395e0a129b1c9785868fa9858.svg
paper_authors.fullname: Egor Cherepanov
paper_authors.type: user
paper_authors.user: avanturist
paper_authors._id: 67583ace95bb6a9c895df273
paper_authors.name: Nikita Kachaev
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-10T15:15:21.788Z
paper_authors._id: 64705ef84be5cf1f3348e283
paper_authors.avatarUrl: /avatars/915875e7c4118098ab460831d5e8ef0e.svg
paper_authors.fullname: Nikita
paper_authors.type: user
paper_authors.user: tttonyalpha
paper_authors._id: 67583ace95bb6a9c895df274
paper_authors.name: Artem Zholus
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:07:54.862Z
paper_authors._id: 6267cdd3f91d1c1633c08bbf
paper_authors.avatarUrl: /avatars/51d7961098f52f39ee406009a12982b8.svg
paper_authors.fullname: Artem Zholus
paper_authors.type: user
paper_authors.user: artemZholus
paper_authors._id: 67583ace95bb6a9c895df275
paper_authors.name: Alexey K. Kovalev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:07:49.350Z
paper_authors._id: 64198f70ed725fef6442b37e
paper_authors.avatarUrl: /avatars/580ab07a3067a9deb2977b0894226fe3.svg
paper_authors.fullname: Alexey Kovalev
paper_authors.type: user
paper_authors.user: AlexeyKov
paper_authors._id: 67583ace95bb6a9c895df276
paper_authors.name: Aleksandr I. Panov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:07:02.724Z
paper_authors._id: 6633243f5ddb7702ad3ec216
paper_authors.avatarUrl: /avatars/0abaf60f1a52e148c3f6fce57eb31eb4.svg
paper_authors.fullname: Aleksandr Panov
paper_authors.type: user
paper_authors.user: grafft
paper_title: Unraveling the Complexity of Memory in RL Agents: an Approach for
    Classification and Evaluation
paper_summary: The incorporation of memory into agents is essential for numerous tasks
  within the domain of Reinforcement Learning (RL). In particular, memory is
  paramount for tasks that require the utilization of past information,
  adaptation to novel environments, and improved sample efficiency. However, the
  term ``memory'' encompasses a wide range of concepts, which, coupled with the
  lack of a unified methodology for validating an agent's memory, leads to
  erroneous judgments about agents' memory capabilities and prevents objective
  comparison with other memory-enhanced agents. This paper aims to streamline the
  concept of memory in RL by providing practical precise definitions of agent
  memory types, such as long-term versus short-term memory and declarative versus
  procedural memory, inspired by cognitive science. Using these definitions, we
  categorize different classes of agent memory, propose a robust experimental
  methodology for evaluating the memory capabilities of RL agents, and
  standardize evaluations. Furthermore, we empirically demonstrate the importance
  of adhering to the proposed methodology when evaluating different types of
  agent memory by conducting experiments with different RL agents and what its
  violation leads to.
paper_ai_keywords: long-term memory