title: SRMT: Shared Memory for Multi-agent Lifelong Pathfinding
summary: Multi-agent reinforcement learning (MARL) demonstrates significant progress
  in solving cooperative and competitive multi-agent problems in various
  environments. One of the principal challenges in MARL is the need for explicit
  prediction of the agents' behavior to achieve cooperation. To resolve this
  issue, we propose the Shared Recurrent Memory Transformer (SRMT) which extends
  memory transformers to multi-agent settings by pooling and globally
  broadcasting individual working memories, enabling agents to exchange
  information implicitly and coordinate their actions. We evaluate SRMT on the
  Partially Observable Multi-Agent Pathfinding problem in a toy Bottleneck
  navigation task that requires agents to pass through a narrow corridor and on a
  POGEMA benchmark set of tasks. In the Bottleneck task, SRMT consistently
  outperforms a variety of reinforcement learning baselines, especially under
  sparse rewards, and generalizes effectively to longer corridors than those seen
  during training. On POGEMA maps, including Mazes, Random, and MovingAI, SRMT is
  competitive with recent MARL, hybrid, and planning-based algorithms. These
  results suggest that incorporating shared recurrent memory into the
  transformer-based architectures can enhance coordination in decentralized
  multi-agent systems. The source code for training and evaluation is available
  on GitHub: https://github.com/Aloriosa/srmt.
paper_authors._id: 67933d69b843fda452c689dd
paper_authors.name: Alsu Sagirova
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T09:07:49.036Z
paper_authors._id: 65c0db0fbda79a18292dfbb7
paper_authors.avatarUrl: /avatars/1201b8282664c2d8c18beaba2396c03b.svg
paper_authors.fullname: Alsu Sagirova
paper_authors.type: user
paper_authors.user: alsu-sagirova
paper_authors._id: 67933d69b843fda452c689de
paper_authors.name: Yuri Kuratov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-26T11:38:38.386Z
paper_authors._id: 618b9540682ec1c38327e586
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/618b9540682ec1c38327e586/v_ZBkfh8O9Zh6C2YQpuBX.jpeg
paper_authors.fullname: Yury Kuratov
paper_authors.type: user
paper_authors.user: yurakuratov
paper_authors._id: 67933d69b843fda452c689df
paper_authors.name: Mikhail Burtsev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:07:03.954Z
paper_authors._id: 639c6e978a34ed9a404c6a7b
paper_authors.avatarUrl: /avatars/c98ca8c9f9ed8509c2f1bb6aa994fd57.svg
paper_authors.fullname: MIKHAIL BURTSEV
paper_authors.type: user
paper_authors.user: mbur
paper_title: SRMT: Shared Memory for Multi-agent Lifelong Pathfinding
paper_summary: Multi-agent reinforcement learning (MARL) demonstrates significant progress
  in solving cooperative and competitive multi-agent problems in various
  environments. One of the principal challenges in MARL is the need for explicit
  prediction of the agents' behavior to achieve cooperation. To resolve this
  issue, we propose the Shared Recurrent Memory Transformer (SRMT) which extends
  memory transformers to multi-agent settings by pooling and globally
  broadcasting individual working memories, enabling agents to exchange
  information implicitly and coordinate their actions. We evaluate SRMT on the
  Partially Observable Multi-Agent Pathfinding problem in a toy Bottleneck
  navigation task that requires agents to pass through a narrow corridor and on a
  POGEMA benchmark set of tasks. In the Bottleneck task, SRMT consistently
  outperforms a variety of reinforcement learning baselines, especially under
  sparse rewards, and generalizes effectively to longer corridors than those seen
  during training. On POGEMA maps, including Mazes, Random, and MovingAI, SRMT is
  competitive with recent MARL, hybrid, and planning-based algorithms. These
  results suggest that incorporating shared recurrent memory into the
  transformer-based architectures can enhance coordination in decentralized
  multi-agent systems. The source code for training and evaluation is available
  on GitHub: https://github.com/Aloriosa/srmt.
paper_ai_keywords: Multi-agent reinforcement learning (MARL) Shared Recurrent Memory Transformer (SRMT) Memory transformers Partially Observed Generative Multi-Agent Pathfinding (POGEMA) Bottleneck navigation task Neural network benchmarks Decentralized multi-agent systems