Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with
  Reinforcement Learning
Memory is crucial for enabling agents to tackle complex tasks with temporal
and spatial dependencies. While many reinforcement learning (RL) algorithms
incorporate memory, the field lacks a universal benchmark to assess an agent's
memory capabilities across diverse scenarios. This gap is particularly evident
in tabletop robotic manipulation, where memory is essential for solving tasks
with partial observability and ensuring robust performance, yet no standardized
benchmarks exist. To address this, we introduce MIKASA (Memory-Intensive Skills
Assessment Suite for Agents), a comprehensive benchmark for memory RL, with
three key contributions: (1) we propose a comprehensive classification
framework for memory-intensive RL tasks, (2) we collect MIKASA-Base - a unified
benchmark that enables systematic evaluation of memory-enhanced agents across
diverse scenarios, and (3) we develop MIKASA-Robo - a novel benchmark of 32
carefully designed memory-intensive tasks that assess memory capabilities in
tabletop robotic manipulation. Our contributions establish a unified framework
for advancing memory RL research, driving the development of more reliable
systems for real-world applications. The code is available at
https://sites.google.com/view/memorybenchrobots/.
67b478517fa6ecaa21d1498d
Egor Cherepanov
claimed_verified
2025-02-18T16:39:34.993Z
6668687caee0993c95b0eb81
/avatars/301fe1f395e0a129b1c9785868fa9858.svg
Egor Cherepanov
user
avanturist
67b478517fa6ecaa21d1498e
Nikita Kachaev
67b478517fa6ecaa21d1498f
Alexey K. Kovalev
67b478517fa6ecaa21d14990
Aleksandr I. Panov
Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with
  Reinforcement Learning
Memory is crucial for enabling agents to tackle complex tasks with temporal
and spatial dependencies. While many reinforcement learning (RL) algorithms
incorporate memory, the field lacks a universal benchmark to assess an agent's
memory capabilities across diverse scenarios. This gap is particularly evident
in tabletop robotic manipulation, where memory is essential for solving tasks
with partial observability and ensuring robust performance, yet no standardized
benchmarks exist. To address this, we introduce MIKASA (Memory-Intensive Skills
Assessment Suite for Agents), a comprehensive benchmark for memory RL, with
three key contributions: (1) we propose a comprehensive classification
framework for memory-intensive RL tasks, (2) we collect MIKASA-Base - a unified
benchmark that enables systematic evaluation of memory-enhanced agents across
diverse scenarios, and (3) we develop MIKASA-Robo - a novel benchmark of 32
carefully designed memory-intensive tasks that assess memory capabilities in
tabletop robotic manipulation. Our contributions establish a unified framework
for advancing memory RL research, driving the development of more reliable
systems for real-world applications. The code is available at
https://sites.google.com/view/memorybenchrobots/.
MIKASA is introduced as a comprehensive benchmark suite for memory-enhanced reinforcement learning, specifically targeting tabletop robotic manipulation tasks to assess and advance memory capabilities.
reinforcement learning memory RL MIKASA memory-intensive RL tasks MIKASA-Base MIKASA-Robo partial observability system reliability real-world applications